{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "0b9ffbb2-8d33-eb0c-5a42-0733bdc3b74b",
    "_uuid": "e14f65723f5ac826104e861c45e58525740d8b2a"
   },
   "source": [
    "# Bikeshare数据集上的特征工程\n",
    "\n",
    "1、\t任务描述\n",
    "请在Capital Bikeshare （美国Washington, D.C.的一个共享单车公司）提供的自行车数据上进行回归分析。根据每天的天气信息，预测该天的单车共享骑行量。\n",
    "\n",
    "原始数据集地址：http://archive.ics.uci.edu/ml/datasets/Bike+Sharing+Dataset\n",
    "1)\t文件说明\n",
    "day.csv: 按天计的单车共享次数（作业只需使用该文件）\n",
    "hour.csv: 按小时计的单车共享次数（无需理会）\n",
    "readme：数据说明文件\n",
    "\n",
    "2)\t字段说明\n",
    "Instant记录号\n",
    "Dteday：日期\n",
    "Season：季节（1=春天、2=夏天、3=秋天、4=冬天）\n",
    "yr：年份，(0: 2011, 1:2012)\n",
    "mnth：月份( 1 to 12)\n",
    "hr：小时 (0 to 23)  （只在hour.csv有，作业忽略此字段）\n",
    "holiday：是否是节假日\n",
    "weekday：星期中的哪天，取值为0～6\n",
    "workingday：是否工作日\n",
    "1=工作日 （是否为工作日，1为工作日，0为非周末或节假日\n",
    "weathersit：天气（1：晴天，多云 ",
    "2：雾天，阴天 ",
    "3：小雪，小雨 ",
    "4：大雨，大雪，大雾）\n",
    "temp：气温摄氏度\n",
    "atemp：体感温度\n",
    "hum：湿度\n",
    "windspeed：风速\n",
    "casual：非注册用户个数\n",
    "registered：注册用户个数\n",
    "cnt：给定日期（天）时间（每小时）总租车人数，响应变量y （cnt = casual + registered）\n",
    "\n",
    "casual、registered和cnt三个特征均为要预测的y，作业里只需对cnt进行预测"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 导入必要的工具包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 364,
   "metadata": {
    "_cell_guid": "2ba3154a-c2aa-3158-1984-63ad2c0c786a",
    "_uuid": "5eb696b95780825e94ddb49787f9fa339fc3833b"
   },
   "outputs": [],
   "source": [
    "import numpy as np  # 矩阵操作\n",
    "import pandas as pd # SQL数据处理\n",
    "\n",
    "import matplotlib.pyplot as plt   #画图\n",
    "import seaborn as sns\n",
    "\n",
    "# 图形出现在Notebook里而不是新窗口\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读入数据\n",
    "\n",
    "数据预处理对训练数据和测试数据需进行同样处理，因此将二者一起读入"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 365,
   "metadata": {
    "_cell_guid": "21fa35be-878b-b4f2-ef6e-68dc070b8bfa",
    "_uuid": "73aee228226be55c0c8a6e4fcbf818c56cd94926"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>dteday</th>\n",
       "      <th>season</th>\n",
       "      <th>yr</th>\n",
       "      <th>mnth</th>\n",
       "      <th>holiday</th>\n",
       "      <th>weekday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>weathersit</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>casual</th>\n",
       "      <th>registered</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2011-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.344167</td>\n",
       "      <td>0.363625</td>\n",
       "      <td>0.805833</td>\n",
       "      <td>0.160446</td>\n",
       "      <td>331</td>\n",
       "      <td>654</td>\n",
       "      <td>985</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2011-01-02</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.363478</td>\n",
       "      <td>0.353739</td>\n",
       "      <td>0.696087</td>\n",
       "      <td>0.248539</td>\n",
       "      <td>131</td>\n",
       "      <td>670</td>\n",
       "      <td>801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>2011-01-03</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.196364</td>\n",
       "      <td>0.189405</td>\n",
       "      <td>0.437273</td>\n",
       "      <td>0.248309</td>\n",
       "      <td>120</td>\n",
       "      <td>1229</td>\n",
       "      <td>1349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>2011-01-04</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.200000</td>\n",
       "      <td>0.212122</td>\n",
       "      <td>0.590435</td>\n",
       "      <td>0.160296</td>\n",
       "      <td>108</td>\n",
       "      <td>1454</td>\n",
       "      <td>1562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>2011-01-05</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.226957</td>\n",
       "      <td>0.229270</td>\n",
       "      <td>0.436957</td>\n",
       "      <td>0.186900</td>\n",
       "      <td>82</td>\n",
       "      <td>1518</td>\n",
       "      <td>1600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   instant      dteday  season  yr  mnth  holiday  weekday  workingday  \\\n",
       "0        1  2011-01-01       1   0     1        0        6           0   \n",
       "1        2  2011-01-02       1   0     1        0        0           0   \n",
       "2        3  2011-01-03       1   0     1        0        1           1   \n",
       "3        4  2011-01-04       1   0     1        0        2           1   \n",
       "4        5  2011-01-05       1   0     1        0        3           1   \n",
       "\n",
       "   weathersit      temp     atemp       hum  windspeed  casual  registered  \\\n",
       "0           2  0.344167  0.363625  0.805833   0.160446     331         654   \n",
       "1           2  0.363478  0.353739  0.696087   0.248539     131         670   \n",
       "2           1  0.196364  0.189405  0.437273   0.248309     120        1229   \n",
       "3           1  0.200000  0.212122  0.590435   0.160296     108        1454   \n",
       "4           1  0.226957  0.229270  0.436957   0.186900      82        1518   \n",
       "\n",
       "    cnt  \n",
       "0   985  \n",
       "1   801  \n",
       "2  1349  \n",
       "3  1562  \n",
       "4  1600  "
      ]
     },
     "execution_count": 365,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 读入数据\n",
    "train = pd.read_csv(\"day.csv\")\n",
    "train.head()\n",
    "\n",
    "# Convert \"object\" to \"daytime\"\n",
    "#train['dteday'] = pd.to_datetime(train['dteday'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 366,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 731 entries, 0 to 730\n",
      "Data columns (total 16 columns):\n",
      "instant       731 non-null int64\n",
      "dteday        731 non-null object\n",
      "season        731 non-null int64\n",
      "yr            731 non-null int64\n",
      "mnth          731 non-null int64\n",
      "holiday       731 non-null int64\n",
      "weekday       731 non-null int64\n",
      "workingday    731 non-null int64\n",
      "weathersit    731 non-null int64\n",
      "temp          731 non-null float64\n",
      "atemp         731 non-null float64\n",
      "hum           731 non-null float64\n",
      "windspeed     731 non-null float64\n",
      "casual        731 non-null int64\n",
      "registered    731 non-null int64\n",
      "cnt           731 non-null int64\n",
      "dtypes: float64(4), int64(11), object(1)\n",
      "memory usage: 91.5+ KB\n"
     ]
    }
   ],
   "source": [
    "train.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 367,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "instant       0\n",
       "dteday        0\n",
       "season        0\n",
       "yr            0\n",
       "mnth          0\n",
       "holiday       0\n",
       "weekday       0\n",
       "workingday    0\n",
       "weathersit    0\n",
       "temp          0\n",
       "atemp         0\n",
       "hum           0\n",
       "windspeed     0\n",
       "casual        0\n",
       "registered    0\n",
       "cnt           0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 367,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "### 查看是否有空值\n",
    "train.isnull().sum() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "没有缺失数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据探索"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 368,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>season</th>\n",
       "      <th>yr</th>\n",
       "      <th>mnth</th>\n",
       "      <th>holiday</th>\n",
       "      <th>weekday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>weathersit</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>casual</th>\n",
       "      <th>registered</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "      <td>731.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>366.000000</td>\n",
       "      <td>2.496580</td>\n",
       "      <td>0.500684</td>\n",
       "      <td>6.519836</td>\n",
       "      <td>0.028728</td>\n",
       "      <td>2.997264</td>\n",
       "      <td>0.683995</td>\n",
       "      <td>1.395349</td>\n",
       "      <td>0.495385</td>\n",
       "      <td>0.474354</td>\n",
       "      <td>0.627894</td>\n",
       "      <td>0.190486</td>\n",
       "      <td>848.176471</td>\n",
       "      <td>3656.172367</td>\n",
       "      <td>4504.348837</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>211.165812</td>\n",
       "      <td>1.110807</td>\n",
       "      <td>0.500342</td>\n",
       "      <td>3.451913</td>\n",
       "      <td>0.167155</td>\n",
       "      <td>2.004787</td>\n",
       "      <td>0.465233</td>\n",
       "      <td>0.544894</td>\n",
       "      <td>0.183051</td>\n",
       "      <td>0.162961</td>\n",
       "      <td>0.142429</td>\n",
       "      <td>0.077498</td>\n",
       "      <td>686.622488</td>\n",
       "      <td>1560.256377</td>\n",
       "      <td>1937.211452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.059130</td>\n",
       "      <td>0.079070</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.022392</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>22.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>183.500000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.337083</td>\n",
       "      <td>0.337842</td>\n",
       "      <td>0.520000</td>\n",
       "      <td>0.134950</td>\n",
       "      <td>315.500000</td>\n",
       "      <td>2497.000000</td>\n",
       "      <td>3152.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>366.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.498333</td>\n",
       "      <td>0.486733</td>\n",
       "      <td>0.626667</td>\n",
       "      <td>0.180975</td>\n",
       "      <td>713.000000</td>\n",
       "      <td>3662.000000</td>\n",
       "      <td>4548.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>548.500000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.655417</td>\n",
       "      <td>0.608602</td>\n",
       "      <td>0.730209</td>\n",
       "      <td>0.233214</td>\n",
       "      <td>1096.000000</td>\n",
       "      <td>4776.500000</td>\n",
       "      <td>5956.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>731.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>0.861667</td>\n",
       "      <td>0.840896</td>\n",
       "      <td>0.972500</td>\n",
       "      <td>0.507463</td>\n",
       "      <td>3410.000000</td>\n",
       "      <td>6946.000000</td>\n",
       "      <td>8714.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          instant      season          yr        mnth     holiday     weekday  \\\n",
       "count  731.000000  731.000000  731.000000  731.000000  731.000000  731.000000   \n",
       "mean   366.000000    2.496580    0.500684    6.519836    0.028728    2.997264   \n",
       "std    211.165812    1.110807    0.500342    3.451913    0.167155    2.004787   \n",
       "min      1.000000    1.000000    0.000000    1.000000    0.000000    0.000000   \n",
       "25%    183.500000    2.000000    0.000000    4.000000    0.000000    1.000000   \n",
       "50%    366.000000    3.000000    1.000000    7.000000    0.000000    3.000000   \n",
       "75%    548.500000    3.000000    1.000000   10.000000    0.000000    5.000000   \n",
       "max    731.000000    4.000000    1.000000   12.000000    1.000000    6.000000   \n",
       "\n",
       "       workingday  weathersit        temp       atemp         hum   windspeed  \\\n",
       "count  731.000000  731.000000  731.000000  731.000000  731.000000  731.000000   \n",
       "mean     0.683995    1.395349    0.495385    0.474354    0.627894    0.190486   \n",
       "std      0.465233    0.544894    0.183051    0.162961    0.142429    0.077498   \n",
       "min      0.000000    1.000000    0.059130    0.079070    0.000000    0.022392   \n",
       "25%      0.000000    1.000000    0.337083    0.337842    0.520000    0.134950   \n",
       "50%      1.000000    1.000000    0.498333    0.486733    0.626667    0.180975   \n",
       "75%      1.000000    2.000000    0.655417    0.608602    0.730209    0.233214   \n",
       "max      1.000000    3.000000    0.861667    0.840896    0.972500    0.507463   \n",
       "\n",
       "            casual   registered          cnt  \n",
       "count   731.000000   731.000000   731.000000  \n",
       "mean    848.176471  3656.172367  4504.348837  \n",
       "std     686.622488  1560.256377  1937.211452  \n",
       "min       2.000000    20.000000    22.000000  \n",
       "25%     315.500000  2497.000000  3152.000000  \n",
       "50%     713.000000  3662.000000  4548.000000  \n",
       "75%    1096.000000  4776.500000  5956.000000  \n",
       "max    3410.000000  6946.000000  8714.000000  "
      ]
     },
     "execution_count": 368,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#对数据值型特征，用常用统计量观察其分布\n",
    "train.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 369,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "season属性的不同取值和出现的次数\n",
      "3    188\n",
      "2    184\n",
      "1    181\n",
      "4    178\n",
      "Name: season, dtype: int64\n",
      "\n",
      "mnth属性的不同取值和出现的次数\n",
      "12    62\n",
      "10    62\n",
      "8     62\n",
      "7     62\n",
      "5     62\n",
      "3     62\n",
      "1     62\n",
      "11    60\n",
      "9     60\n",
      "6     60\n",
      "4     60\n",
      "2     57\n",
      "Name: mnth, dtype: int64\n",
      "\n",
      "weathersit属性的不同取值和出现的次数\n",
      "1    463\n",
      "2    247\n",
      "3     21\n",
      "Name: weathersit, dtype: int64\n",
      "\n",
      "weekday属性的不同取值和出现的次数\n",
      "6    105\n",
      "1    105\n",
      "0    105\n",
      "5    104\n",
      "4    104\n",
      "3    104\n",
      "2    104\n",
      "Name: weekday, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "#对类别型特征，观察其取值范围及直方图\n",
    "categorical_features = ['season','mnth','weathersit','weekday']\n",
    "for col in categorical_features:\n",
    "    print(\"\\n%s属性的不同取值和出现的次数\"%col)\n",
    "    print(train[col].value_counts())\n",
    "    train[col] = train[col].astype('object')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "类别型特征的取值不多 类别型特征可以采用独热编码（One hot encoding）/哑编码"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 特征工程"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 370,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>mnth_6</th>\n",
       "      <th>...</th>\n",
       "      <th>weathersit_1</th>\n",
       "      <th>weathersit_2</th>\n",
       "      <th>weathersit_3</th>\n",
       "      <th>weekday_0</th>\n",
       "      <th>weekday_1</th>\n",
       "      <th>weekday_2</th>\n",
       "      <th>weekday_3</th>\n",
       "      <th>weekday_4</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  mnth_4  \\\n",
       "0         1         0         0         0       1       0       0       0   \n",
       "1         1         0         0         0       1       0       0       0   \n",
       "2         1         0         0         0       1       0       0       0   \n",
       "3         1         0         0         0       1       0       0       0   \n",
       "4         1         0         0         0       1       0       0       0   \n",
       "\n",
       "   mnth_5  mnth_6    ...      weathersit_1  weathersit_2  weathersit_3  \\\n",
       "0       0       0    ...                 0             1             0   \n",
       "1       0       0    ...                 0             1             0   \n",
       "2       0       0    ...                 1             0             0   \n",
       "3       0       0    ...                 1             0             0   \n",
       "4       0       0    ...                 1             0             0   \n",
       "\n",
       "   weekday_0  weekday_1  weekday_2  weekday_3  weekday_4  weekday_5  weekday_6  \n",
       "0          0          0          0          0          0          0          1  \n",
       "1          1          0          0          0          0          0          0  \n",
       "2          0          1          0          0          0          0          0  \n",
       "3          0          0          1          0          0          0          0  \n",
       "4          0          0          0          1          0          0          0  \n",
       "\n",
       "[5 rows x 26 columns]"
      ]
     },
     "execution_count": 370,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "categorical_features = ['season','mnth','weathersit','weekday']\n",
    "x_train_cat = train[categorical_features]\n",
    "x_train_cat = pd.get_dummies(x_train_cat)\n",
    "x_train_cat.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 371,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.355170</td>\n",
       "      <td>0.373517</td>\n",
       "      <td>0.828620</td>\n",
       "      <td>0.284606</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.379232</td>\n",
       "      <td>0.360541</td>\n",
       "      <td>0.715771</td>\n",
       "      <td>0.466215</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.171000</td>\n",
       "      <td>0.144830</td>\n",
       "      <td>0.449638</td>\n",
       "      <td>0.465740</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.175530</td>\n",
       "      <td>0.174649</td>\n",
       "      <td>0.607131</td>\n",
       "      <td>0.284297</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.209120</td>\n",
       "      <td>0.197158</td>\n",
       "      <td>0.449313</td>\n",
       "      <td>0.339143</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       temp     atemp       hum  windspeed\n",
       "0  0.355170  0.373517  0.828620   0.284606\n",
       "1  0.379232  0.360541  0.715771   0.466215\n",
       "2  0.171000  0.144830  0.449638   0.465740\n",
       "3  0.175530  0.174649  0.607131   0.284297\n",
       "4  0.209120  0.197158  0.449313   0.339143"
      ]
     },
     "execution_count": 371,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#数值型变量预处理\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "# MinMaxScaler rescales the data set such that all feature values are in the range [0, 1]\n",
    "mn_x = MinMaxScaler() \n",
    "numerical_features = ['temp','atemp','hum','windspeed']\n",
    "temp = mn_x.fit_transform(train[numerical_features])\n",
    "\n",
    "x_train_num = pd.DataFrame(data=temp, columns=numerical_features, index =train.index)\n",
    "x_train_num.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 372,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>mnth_6</th>\n",
       "      <th>...</th>\n",
       "      <th>weekday_3</th>\n",
       "      <th>weekday_4</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.355170</td>\n",
       "      <td>0.373517</td>\n",
       "      <td>0.828620</td>\n",
       "      <td>0.284606</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.379232</td>\n",
       "      <td>0.360541</td>\n",
       "      <td>0.715771</td>\n",
       "      <td>0.466215</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.171000</td>\n",
       "      <td>0.144830</td>\n",
       "      <td>0.449638</td>\n",
       "      <td>0.465740</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.175530</td>\n",
       "      <td>0.174649</td>\n",
       "      <td>0.607131</td>\n",
       "      <td>0.284297</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.209120</td>\n",
       "      <td>0.197158</td>\n",
       "      <td>0.449313</td>\n",
       "      <td>0.339143</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 32 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  mnth_4  \\\n",
       "0         1         0         0         0       1       0       0       0   \n",
       "1         1         0         0         0       1       0       0       0   \n",
       "2         1         0         0         0       1       0       0       0   \n",
       "3         1         0         0         0       1       0       0       0   \n",
       "4         1         0         0         0       1       0       0       0   \n",
       "\n",
       "   mnth_5  mnth_6     ...      weekday_3  weekday_4  weekday_5  weekday_6  \\\n",
       "0       0       0     ...              0          0          0          1   \n",
       "1       0       0     ...              0          0          0          0   \n",
       "2       0       0     ...              0          0          0          0   \n",
       "3       0       0     ...              0          0          0          0   \n",
       "4       0       0     ...              1          0          0          0   \n",
       "\n",
       "       temp     atemp       hum  windspeed  holiday  workingday  \n",
       "0  0.355170  0.373517  0.828620   0.284606        0           0  \n",
       "1  0.379232  0.360541  0.715771   0.466215        0           0  \n",
       "2  0.171000  0.144830  0.449638   0.465740        0           1  \n",
       "3  0.175530  0.174649  0.607131   0.284297        0           1  \n",
       "4  0.209120  0.197158  0.449313   0.339143        0           1  \n",
       "\n",
       "[5 rows x 32 columns]"
      ]
     },
     "execution_count": 372,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Join categorical and numerical features\n",
    "x_train = pd.concat([x_train_cat, x_train_num, train['holiday'],  train['workingday']], axis = 1, ignore_index=False)\n",
    "x_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 373,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>...</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>yr</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.355170</td>\n",
       "      <td>0.373517</td>\n",
       "      <td>0.828620</td>\n",
       "      <td>0.284606</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>985</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.379232</td>\n",
       "      <td>0.360541</td>\n",
       "      <td>0.715771</td>\n",
       "      <td>0.466215</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.171000</td>\n",
       "      <td>0.144830</td>\n",
       "      <td>0.449638</td>\n",
       "      <td>0.465740</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.175530</td>\n",
       "      <td>0.174649</td>\n",
       "      <td>0.607131</td>\n",
       "      <td>0.284297</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.209120</td>\n",
       "      <td>0.197158</td>\n",
       "      <td>0.449313</td>\n",
       "      <td>0.339143</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   instant  season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  \\\n",
       "0        1         1         0         0         0       1       0       0   \n",
       "1        2         1         0         0         0       1       0       0   \n",
       "2        3         1         0         0         0       1       0       0   \n",
       "3        4         1         0         0         0       1       0       0   \n",
       "4        5         1         0         0         0       1       0       0   \n",
       "\n",
       "   mnth_4  mnth_5  ...   weekday_5  weekday_6      temp     atemp       hum  \\\n",
       "0       0       0  ...           0          1  0.355170  0.373517  0.828620   \n",
       "1       0       0  ...           0          0  0.379232  0.360541  0.715771   \n",
       "2       0       0  ...           0          0  0.171000  0.144830  0.449638   \n",
       "3       0       0  ...           0          0  0.175530  0.174649  0.607131   \n",
       "4       0       0  ...           0          0  0.209120  0.197158  0.449313   \n",
       "\n",
       "   windspeed  holiday  workingday  yr   cnt  \n",
       "0   0.284606        0           0   0   985  \n",
       "1   0.466215        0           0   0   801  \n",
       "2   0.465740        0           1   0  1349  \n",
       "3   0.284297        0           1   0  1562  \n",
       "4   0.339143        0           1   0  1600  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 373,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "FE_train = pd.concat([train['instant'], x_train,  train['yr'],train['cnt']], axis = 1)\n",
    "FE_train.to_csv('FE_day.csv', index=False)\n",
    "FE_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 374,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 731 entries, 0 to 730\n",
      "Data columns (total 35 columns):\n",
      "instant         731 non-null int64\n",
      "season_1        731 non-null uint8\n",
      "season_2        731 non-null uint8\n",
      "season_3        731 non-null uint8\n",
      "season_4        731 non-null uint8\n",
      "mnth_1          731 non-null uint8\n",
      "mnth_2          731 non-null uint8\n",
      "mnth_3          731 non-null uint8\n",
      "mnth_4          731 non-null uint8\n",
      "mnth_5          731 non-null uint8\n",
      "mnth_6          731 non-null uint8\n",
      "mnth_7          731 non-null uint8\n",
      "mnth_8          731 non-null uint8\n",
      "mnth_9          731 non-null uint8\n",
      "mnth_10         731 non-null uint8\n",
      "mnth_11         731 non-null uint8\n",
      "mnth_12         731 non-null uint8\n",
      "weathersit_1    731 non-null uint8\n",
      "weathersit_2    731 non-null uint8\n",
      "weathersit_3    731 non-null uint8\n",
      "weekday_0       731 non-null uint8\n",
      "weekday_1       731 non-null uint8\n",
      "weekday_2       731 non-null uint8\n",
      "weekday_3       731 non-null uint8\n",
      "weekday_4       731 non-null uint8\n",
      "weekday_5       731 non-null uint8\n",
      "weekday_6       731 non-null uint8\n",
      "temp            731 non-null float64\n",
      "atemp           731 non-null float64\n",
      "hum             731 non-null float64\n",
      "windspeed       731 non-null float64\n",
      "holiday         731 non-null int64\n",
      "workingday      731 non-null int64\n",
      "yr              731 non-null int64\n",
      "cnt             731 non-null int64\n",
      "dtypes: float64(4), int64(5), uint8(26)\n",
      "memory usage: 70.0 KB\n"
     ]
    }
   ],
   "source": [
    "FE_train.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 数据准备"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 375,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 从原始数据中分离输入特征x和输出y\n",
    "y = FE_train['cnt'].values\n",
    "X = FE_train.drop(['cnt', 'instant'], axis = 1) \n",
    "\n",
    "#用于后续显示权重系数对应的特征\n",
    "columns = X.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练集，测试集分离"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 376,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将数据分割成训练数据与测试数据\n",
    "from sklearn.cross_validation import train_test_split\n",
    "\n",
    "# 随机采样80%的数据作为训练样本，其余的20%作为测试样本\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 26)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据标准化处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 377,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/sddy9/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:475: DataConversionWarning: Data with input dtype int64 was converted to float64 by StandardScaler.\n",
      "  warnings.warn(msg, DataConversionWarning)\n",
      "/Users/sddy9/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:475: DataConversionWarning: Data with input dtype int64 was converted to float64 by StandardScaler.\n",
      "  warnings.warn(msg, DataConversionWarning)\n",
      "/Users/sddy9/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:475: DataConversionWarning: Data with input dtype int64 was converted to float64 by StandardScaler.\n",
      "  warnings.warn(msg, DataConversionWarning)\n"
     ]
    }
   ],
   "source": [
    "# 数据标准化\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "# 初始化标准化函数\n",
    "ss_y = StandardScaler()\n",
    "\n",
    "#对y做标准化不是必须\n",
    "#对y标准化的好处是不同问题的w差异不太大，同时正则参数的范围也有限\n",
    "y_train = ss_y.fit_transform(y_train.reshape(-1, 1))\n",
    "y_test = ss_y.transform(y_test.reshape(-1, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 确定模型类型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**1.尝试缺省参数的线性回归/最小二乘线性回归**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 378,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>[81468554938586.86]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>[81468554938586.69]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>[81468554938585.81]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>workingday</td>\n",
       "      <td>[27391236788852.863]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>holiday</td>\n",
       "      <td>[27391236788852.582]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>[26330226186121.223]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>[26330226186121.03]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>temp</td>\n",
       "      <td>[1.6257144485015205]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>yr</td>\n",
       "      <td>[1.0310554943013157]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>atemp</td>\n",
       "      <td>[0.3700555230824113]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>[-0.7869353645955468]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>hum</td>\n",
       "      <td>[-0.9067522720326273]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>[-1061010602731.6381]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>[-1061010602731.6528]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>[-1061010602731.6815]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>[-1061010602731.7129]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>[-1061010602731.742]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_4</td>\n",
       "      <td>[-11647555030435.684]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_3</td>\n",
       "      <td>[-11647555030436.236]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_2</td>\n",
       "      <td>[-11647555030436.273]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>season_1</td>\n",
       "      <td>[-11647555030436.523]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>[-20261068421518.26]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>[-20261068421518.312]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>[-20261068421518.44]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>[-20261068421518.508]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>[-20261068421518.527]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>[-20261068421518.547]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>[-20261068421518.613]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>[-20261068421518.754]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>[-20261068421518.816]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>[-20261068421518.84]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>[-20261068421518.9]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>[-20261068421518.92]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns                   coef\n",
       "16  weathersit_1    [81468554938586.86]\n",
       "17  weathersit_2    [81468554938586.69]\n",
       "18  weathersit_3    [81468554938585.81]\n",
       "31    workingday   [27391236788852.863]\n",
       "30       holiday   [27391236788852.582]\n",
       "25     weekday_6   [26330226186121.223]\n",
       "19     weekday_0    [26330226186121.03]\n",
       "26          temp   [1.6257144485015205]\n",
       "32            yr   [1.0310554943013157]\n",
       "27         atemp   [0.3700555230824113]\n",
       "29     windspeed  [-0.7869353645955468]\n",
       "28           hum  [-0.9067522720326273]\n",
       "24     weekday_5  [-1061010602731.6381]\n",
       "23     weekday_4  [-1061010602731.6528]\n",
       "22     weekday_3  [-1061010602731.6815]\n",
       "20     weekday_1  [-1061010602731.7129]\n",
       "21     weekday_2   [-1061010602731.742]\n",
       "3       season_4  [-11647555030435.684]\n",
       "2       season_3  [-11647555030436.236]\n",
       "1       season_2  [-11647555030436.273]\n",
       "0       season_1  [-11647555030436.523]\n",
       "8         mnth_5   [-20261068421518.26]\n",
       "12        mnth_9  [-20261068421518.312]\n",
       "7         mnth_4   [-20261068421518.44]\n",
       "6         mnth_3  [-20261068421518.508]\n",
       "11        mnth_8  [-20261068421518.527]\n",
       "9         mnth_6  [-20261068421518.547]\n",
       "13       mnth_10  [-20261068421518.613]\n",
       "10        mnth_7  [-20261068421518.754]\n",
       "4         mnth_1  [-20261068421518.816]\n",
       "5         mnth_2   [-20261068421518.84]\n",
       "14       mnth_11    [-20261068421518.9]\n",
       "15       mnth_12   [-20261068421518.92]"
      ]
     },
     "execution_count": 378,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 线性回归\n",
    "#class sklearn.linear_model.LinearRegression(fit_intercept=True, normalize=False, copy_X=True, n_jobs=1)\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "# 创建线性回归模型，使用初始化设置\n",
    "lr = LinearRegression()\n",
    "\n",
    "# 训练模型参数\n",
    "lr.fit(X_train, y_train)\n",
    "\n",
    "# 预测\n",
    "y_test_pred_lr = lr.predict(X_test)\n",
    "y_train_pred_lr = lr.predict(X_train)\n",
    "\n",
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(columns), \"coef\":list((lr.coef_.T))})\n",
    "fs.sort_values(by=['coef'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**模型评价**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 379,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of LinearRegression on test is 0.41284872438380044\n",
      "The RMSE of LinearRegression on train is 0.39314483853631543\n"
     ]
    }
   ],
   "source": [
    "# Import necessary modules -- 评价回归预测模型的性能\n",
    "# from sklearn.metrics import r2_score  \n",
    "from sklearn.metrics import mean_squared_error \n",
    "\n",
    "# 使用 RMSE 评价模型在测试集和训练集上的性能，并输出评估结果\n",
    "\n",
    "# 测试集\n",
    "rmse_test = np.sqrt(mean_squared_error(y_test, y_test_pred_lr))\n",
    "print(\"The RMSE of LinearRegression on test is {}\".format(rmse_test))\n",
    "\n",
    "# 训练集\n",
    "rmse_train = np.sqrt(mean_squared_error(y_train, y_train_pred_lr))\n",
    "print(\"The RMSE of LinearRegression on train is {}\".format(rmse_train))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 380,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAekAAAFsCAYAAADos0H8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHclJREFUeJzt3Xuc1XW97/HXRxihLQqhoCgZWFpeARsNMo2i1B0m7seurOMFj6hll4e7ThfNU5G7dnn01M6jdTa7i5iWKGcb7trtrYdkm57UwMgbFmRaKMFIeUEzBT/nj/UbGmGGWcysYb4z6/V8POax1u+yfr/Pd/0Y3uv7+33ntyIzkSRJ5dmpvwuQJEmdM6QlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdJqehFxf0RM7+86+lNE/E1E/C4iNkTElB243w0RsV8Xy86IiNsatJ+HI+KtjdiWtCMZ0hrUOvvPecv//DPz4Mxc0s12JkRERsTQPiq1v10KfCgzR2Tmz7dcWLX9mSpUH42IL0fEkN7utNrfQ73djjRYGdJSAQoI/1cC93ezzqTMHAG8CTgZOLPPq5KanCGtptextx0RR0bE0oh4KiLWRsSXq9VurR6fqHqT0yJip4j47xHxSESsi4irImJkh+2eXi1bHxGf3mI/cyNiYURcHRFPAWdU+/5pRDwREWsi4vKI2LnD9jIiPhARKyPi6Yj4+4h4VfWapyLiuo7rb9HGTmuNiGERsQEYAvwiIn7d3fuVmauA24HJHbY/MiK+WdX9aER8vr2nHRGvjoj/jIgnI+LxiFiwRZteXT3fPSJurNpyF/CqDuttdSYjIpZExFnV81dFxI+r9/rxiLgmIkZ18V50dYyl4hjS0kt9FfhqZu5GLSSuq+YfUz2Oqk7R/hQ4o/p5M7AfMAK4HCAiDgK+BpwCjANGAvtssa9ZwEJgFHANsAn4CLAHMA2YAXxgi9ccD7wOmAp8AphX7eMVwCHAe7toV6e1Zuafq94x1HrKr+r85X8REa8FjgZWdZg9H9gIvBqYAhwLnFUt+3vgJuDlwHjgf3Wx6SuA56i9X2eyfT31AL4I7A0cSO39mNvFul0dY6k4hrSawfer3ukTEfEEtfDsygvAqyNij8zckJl3bGPdU4AvZ+ZDmbkBuAB4T9Xbeyfwr5l5W2Y+D3wG2PJG+T/NzO9n5ouZ+afMXJaZd2Tmxsx8GPgnaqeWO7o4M5/KzPuB+4Cbqv0/CfyIWkBub631ujsingFWAEuo3seI2BP4a+DvMvOZzFwHfAV4T/W6F6idTt87M5/LzK0Gg1W97r8FPlNt4z5qwV+XzFyVmTdXHzragC+z9XvXbnuOsdSvDGk1g5Myc1T7D1v3TjuaAxwAPBgRP4uIE7ax7t7AIx2mHwGGAntWy37XviAznwXWb/H633WciIgDIuIHEfH76hT4P1DrVXe0tsPzP3UyPYLObavWeh1ebf9k4PXALtX8VwItwJoOH4T+CRhbLf8EtZ7uXVEbSd9ZD3lMVU/H9+SRTtbrVESMjYhrq1PtTwFXs/V71257jrHUrwxpqYPMXJmZ76UWMBcDCyNiF7buBQM8Ri2g2u1L7ZTvWmANtVO7AETEy4Ddt9zdFtNfBx4E9q9OxX6KWrg1wrZqrVvWXAf8lNrZAagF65+BPTp8GNotMw+uXvP7zDw7M/cG3gd8rf06dAdtVT2v2KLGds9Uj3/VYd5eHZ5/kdr7eVj13p1KF+/dNo6xVBxDWuogIk6NiDGZ+SLwRDV7E7UQeZHa9dx23wM+EhETI2IEtZ7vgszcSO1a8zsi4g3VYK7P0X3g7go8BWyorvue27CGbbvWnvgScE5E7JWZa6hdc/6fEbFbNUjtVRHxJoCIeFdEtH9g+SO1MN3UcWOZuQn4F2BuRPxVdU1/doflbcCjwKkRMaTqjXe8fr4rsIHawL59gI93Vfg2jrFUHENaeqnjgfurEc9fBd5TXUd9FvgCcHt1Sncq8C3gO9RGfv+G2qCnDwNU14w/DFxLrVf9NLCOWo+zKx8D/ku17j8DC7ax7vbqstaeyMx7gf/kL2F4OrAz8AC1IF5IbQAYwBHAndV7eiNwXmb+ppPNfoja6fTfA1cC395i+dnV/tYDBwP/r8Oyz1E7Hf8k8ENqgd+VTo/xtlss9Y/I7OwsnqRGqnqvT1A7ld1ZQEnSVuxJS30kIt5Rnbrdhdodve4FHu7fqiQNJIa01HdmURuw9RiwP7XTqp66klQ3T3dLklQoe9KSJBVqh97Uf4899sgJEybsyF1KklSUZcuWPZ6ZY+pZd4eG9IQJE1i6dOmO3KUkSUWJiLrvpufpbkmSCmVIS5JUKENakqRC7dBr0pKkzr3wwgusXr2a557zDqWDxfDhwxk/fjwtLS093oYhLUkFWL16NbvuuisTJkwgolFffqb+kpmsX7+e1atXM3HixB5vx9PdklSA5557jt13392AHiQigt13373XZ0YMaUkqhAE9uDTieBrSkiQVymvSklSguXN3/PaGDBnCoYceysaNG5k4cSLf+c53GDVq1Hbv66yzzuKjH/0oBx100EvmX3nllSxdupTLL798u7cJMGLECDZs2FDXutOnT+fSSy+ltbV187ylS5dy1VVXcdlll/Vo//3BnrQkCYCXvexlLF++nPvuu4/Ro0dzxRVX9Gg73/jGN7YK6BK0trb2eUBv2rSpodszpCVJW5k2bRqPPvro5ulLLrmEI444gsMOO4zPfvazADzzzDPMnDmTSZMmccghh7BgwQKg1ottvwX0t7/9bQ444ADe9KY3cfvtt2/e3hlnnMHChQs3T48YMQKADRs2MGPGDA4//HAOPfRQFi1atFVta9as4ZhjjmHy5Mkccsgh/OQnP6mrTUuWLOGEE04AYO7cuZx55plMnz6d/fbb7yXhffXVV3PkkUcyefJk3ve+920O3nPPPZfW1lYOPvjgze8B1G55fdFFF/HGN76R66+/vq5a6uXpbknSS2zatInFixczZ84cAG666SZWrlzJXXfdRWZy4okncuutt9LW1sbee+/ND3/4QwCefPLJl2xnzZo1fPazn2XZsmWMHDmSN7/5zUyZMmWb+x4+fDg33HADu+22G48//jhTp07lxBNPfMkgrO9+97scd9xxXHjhhWzatIlnn322R+188MEHueWWW3j66ad5zWtew7nnnsuqVatYsGABt99+Oy0tLXzgAx/gmmuu4fTTT+cLX/gCo0ePZtOmTcyYMYN77rmHww47bHPdt912W4/q2BZDWpIEwJ/+9CcmT57Mww8/zOte9zre9ra3AbWQvummmzYH7IYNG1i5ciVHH300H/vYx/jkJz/JCSecwNFHH/2S7d15551Mnz6dMWNqX/h08skn86tf/WqbNWQmn/rUp7j11lvZaaedePTRR1m7di177bXX5nWOOOIIzjzzTF544QVOOukkJk+e3KP2zpw5k2HDhjFs2DDGjh3L2rVrWbx4McuWLeOII47Y/J6MHTsWgOuuu4558+axceNG1qxZwwMPPLA5pE8++eQe1dAdT3dLkoC/XJN+5JFHeP755zdfk85MLrjgApYvX87y5ctZtWoVc+bM4YADDmDZsmUceuihXHDBBVx00UVbbbOrP0MaOnQoL7744ubtP//88wBcc801tLW1sWzZMpYvX86ee+651d8aH3PMMdx6663ss88+nHbaaVx11VU9au+wYcM2Px8yZAgbN24kM5k9e/bmtv7yl79k7ty5/OY3v+HSSy9l8eLF3HPPPcycOfMlde2yyy49qqE79qQl7RCNHq08UPY9EI0cOZLLLruMWbNmce6553Lcccfx6U9/mlNOOYURI0bw6KOP0tLSwsaNGxk9ejSnnnoqI0aM4Morr3zJdl7/+tdz3nnnsX79enbbbTeuv/56Jk2aBNSu4y5btox3v/vdLFq0iBdeeAGonTIfO3YsLS0t3HLLLTzyyNbf6vjII4+wzz77cPbZZ/PMM89w9913c/rppzek7TNmzGDWrFl85CMfYezYsfzhD3/g6aef5qmnnmKXXXZh5MiRrF27lh/96EdMnz69IfvcFkNakgrU3x8spkyZwqRJk7j22ms57bTTWLFiBdOmTQNqg7yuvvpqVq1axcc//nF22mknWlpa+PrXv/6SbYwbN465c+cybdo0xo0bx+GHH755ENbZZ5/NrFmzOPLII5kxY8bmnugpp5zCO97xDlpbW5k8eTKvfe1rt6ptyZIlXHLJJbS0tDBixIgue9IzZ87cfN/sadOm8cEPfrDbdh900EF8/vOf59hjj+XFF1+kpaWFK664gqlTpzJlyhQOPvhg9ttvP4466qj638xeiMzcITsCaG1tzfYRf5Kaiz3pbVuxYgUHHnhgf5ehBuvsuEbEssxs7eIlL+E1aUmSCmVIS5JUKENakgqxIy8/qu814nga0pJUgOHDh7N+/XqDepBo/z7p4cOH92o7ju6WpAKMHz+e1atX09bW1t+lqEGGDx/O+PHje7UNQ1qSCtDS0sLEiRP7uwwVxtPdkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUqLpCOiJGRcTCiHgwIlZExLSIGB0RN0fEyurx5X1drCRJzaTenvRXgX/PzNcCk4AVwPnA4szcH1hcTUuSpAbpNqQjYjfgGOCbAJn5fGY+AcwC5lerzQdO6qsiJUlqRvX0pPcD2oBvR8TPI+IbEbELsGdmrgGoHsd29uKIOCcilkbE0ra2toYVLknSYFdPSA8FDge+nplTgGfYjlPbmTkvM1szs3XMmDE9LFOSpOZTT0ivBlZn5p3V9EJqob02IsYBVI/r+qZESZKaU7chnZm/B34XEa+pZs0AHgBuBGZX82YDi/qkQkmSmtTQOtf7MHBNROwMPAT8V2oBf11EzAF+C7yrb0qUJKk51RXSmbkcaO1k0YzGliNJktp5xzFJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVKih/V2ApDLNndvY9SRtP3vSkiQVypCWJKlQhrQkSYUypCVJKpQDxyQNeg6C00BlT1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSpUXaO7I+Jh4GlgE7AxM1sjYjSwAJgAPAy8OzP/2DdlSpLUfLanJ/3mzJycma3V9PnA4szcH1hcTUuSpAbpzenuWcD86vl84KTelyNJktrVG9IJ3BQRyyLinGrenpm5BqB6HNvZCyPinIhYGhFL29rael+xJElNot47jh2VmY9FxFjg5oh4sN4dZOY8YB5Aa2tr9qBGSZKaUl096cx8rHpcB9wAHAmsjYhxANXjur4qUpKkZtRtSEfELhGxa/tz4FjgPuBGYHa12mxgUV8VKUlSM6rndPeewA0R0b7+dzPz3yPiZ8B1ETEH+C3wrr4rU1Kp/FIKqe90G9KZ+RAwqZP564EZfVGUJEnyjmOSJBXLkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKlTdIR0RQyLi5xHxg2p6YkTcGRErI2JBROzcd2VKktR8tqcnfR6wosP0xcBXMnN/4I/AnEYWJklSs6srpCNiPDAT+EY1HcBbgIXVKvOBk/qiQEmSmlW9Pel/BD4BvFhN7w48kZkbq+nVwD4Nrk2SpKbWbUhHxAnAusxc1nF2J6tmF68/JyKWRsTStra2HpYpSVLzqacnfRRwYkQ8DFxL7TT3PwKjImJotc544LHOXpyZ8zKzNTNbx4wZ04CSJUlqDt2GdGZekJnjM3MC8B7gx5l5CnAL8M5qtdnAoj6rUpKkJtSbv5P+JPDRiFhF7Rr1NxtTkiRJAhja/Sp/kZlLgCXV84eAIxtfkiRJAu84JklSsQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhhvZ3AZJUirlz+2ZdqafsSUuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBWq25COiOERcVdE/CIi7o+Iz1XzJ0bEnRGxMiIWRMTOfV+uJEnNo56e9J+Bt2TmJGAycHxETAUuBr6SmfsDfwTm9F2ZkiQ1n25DOms2VJMt1U8CbwEWVvPnAyf1SYWSJDWpofWsFBFDgGXAq4ErgF8DT2TmxmqV1cA+Xbz2HOAcgH333be39Urqpblz+7sCSfWqa+BYZm7KzMnAeOBI4MDOVuvitfMyszUzW8eMGdPzSiVJajLbNbo7M58AlgBTgVER0d4THw881tjSJElqbvWM7h4TEaOq5y8D3gqsAG4B3lmtNhtY1FdFSpLUjOq5Jj0OmF9dl94JuC4zfxARDwDXRsTngZ8D3+zDOiVJajrdhnRm3gNM6WT+Q9SuT0uSpD7gHcckSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKtTQ/i5AkgaiuXMbu57UGXvSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqH8FixpEPCblqTByZ60JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUqG5DOiJeERG3RMSKiLg/Is6r5o+OiJsjYmX1+PK+L1eSpOZRT096I/DfMvNAYCrwwYg4CDgfWJyZ+wOLq2lJktQg3YZ0Zq7JzLur508DK4B9gFnA/Gq1+cBJfVWkJEnNaLuuSUfEBGAKcCewZ2augVqQA2O7eM05EbE0Ipa2tbX1rlpJkppI3SEdESOA/wP8XWY+Ve/rMnNeZrZmZuuYMWN6UqMkSU2prpCOiBZqAX1NZv5LNXttRIyrlo8D1vVNiZIkNad6RncH8E1gRWZ+ucOiG4HZ1fPZwKLGlydJUvOq5ws2jgJOA+6NiOXVvE8BXwKui4g5wG+Bd/VNiZIkNaduQzozbwOii8UzGluOJElq5x3HJEkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpELV81WVkvrJ3Ln9XYGk/mRPWpKkQhnSkiQVypCWJKlQXpOWpD5U77gCxx+oM/akJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEL5LVhSN7bn24n8xiP1VF/8O9PAZ09akqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqVLchHRHfioh1EXFfh3mjI+LmiFhZPb68b8uUJKn51NOTvhI4fot55wOLM3N/YHE1LUmSGqjbkM7MW4E/bDF7FjC/ej4fOKnBdUmS1PR6ejOTPTNzDUBmromIsV2tGBHnAOcA7Lvvvj3cnSSpnTfNaR59PnAsM+dlZmtmto4ZM6avdydJ0qDR05BeGxHjAKrHdY0rSZIkQc9D+kZgdvV8NrCoMeVIkqR29fwJ1veAnwKviYjVETEH+BLwtohYCbytmpYkSQ3U7cCxzHxvF4tmNLgWSZLUgXcckySpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBWqp9+CJakTfuuQpEayJy1JUqEMaUmSCmVIS5JUKK9Jq2l5/VhS6exJS5JUKENakqRCGdKSJBXKkJYkqVAOHNOg44AwSYOFPWlJkgplSEuSVChDWpKkQnlNWtul3uu9XheWBp/t+b32/4DGsCctSVKhDGlJkgplSEuSVChDWpKkQjlwTP3KgSiS1DV70pIkFcqQliSpUIa0JEmFMqQlSSqUA8cGMe8OJjU3f7cHPnvSkiQVypCWJKlQhrQkSYUa0Nek+/NGGH1xrcdryNvWrO2WBrP+/L95IPyfa09akqRC9SqkI+L4iPhlRKyKiPMbVZQkSepFSEfEEOAK4K+Bg4D3RsRBjSpMkqRm15ue9JHAqsx8KDOfB64FZjWmLEmSFJnZsxdGvBM4PjPPqqZPA16fmR/aYr1zgHOqydcAv+x5uVvZA3i8gdsrXbO1F5qvzbZ38Gu2Ntverb0yM8fUs7HejO6OTuZtlfiZOQ+Y14v9dF1AxNLMbO2LbZeo2doLzddm2zv4NVubbW/v9OZ092rgFR2mxwOP9a4cSZLUrjch/TNg/4iYGBE7A+8BbmxMWZIkqcenuzNzY0R8CPgPYAjwrcy8v2GV1adPTqMXrNnaC83XZts7+DVbm21vL/R44JgkSepb3nFMkqRCGdKSJBVqQIV0RFwSEQ9GxD0RcUNEjOpivUFxu9KIeFdE3B8RL0ZEl0P6I+LhiLg3IpZHxNIdWWOjbUebB8sxHh0RN0fEyurx5V2st6k6vssjYsAN0OzueEXEsIhYUC2/MyIm7PgqG6eO9p4REW0djulZ/VFno0TEtyJiXUTc18XyiIjLqvfjnog4fEfX2Eh1tHd6RDzZ4fh+psc7y8wB8wMcCwytnl8MXNzJOkOAXwP7ATsDvwAO6u/ae9jeA6ndAGYJ0LqN9R4G9ujvendUmwfZMf4fwPnV8/M7+zddLdvQ37X2oo3dHi/gA8D/rp6/B1jQ33X3cXvPAC7v71ob2OZjgMOB+7pY/nbgR9TurzEVuLO/a+7j9k4HftCIfQ2onnRm3pSZG6vJO6j9bfaWBs3tSjNzRWY28g5txauzzYPmGFOre371fD5wUj/W0lfqOV4d34eFwIyI6OyGSQPBYPr3WZfMvBX4wzZWmQVclTV3AKMiYtyOqa7x6mhvwwyokN7CmdQ+mW1pH+B3HaZXV/MGswRuiohl1W1YB7vBdIz3zMw1ANXj2C7WGx4RSyPijogYaEFez/HavE71QfxJYPcdUl3j1fvv82+rU78LI+IVnSwfTAbT72y9pkXELyLiRxFxcE830pvbgvaJiPi/wF6dLLowMxdV61wIbASu6WwTncwr9u/M6mlvHY7KzMciYixwc0Q8WH3SK1ID2jxojvF2bGbf6hjvB/w4Iu7NzF83psI+V8/xGlDHtBv1tOVfge9l5p8j4v3UziK8pc8r6z+D6fjW425q9+feEBFvB74P7N+TDRUX0pn51m0tj4jZwAnAjKxO/m9hQN2utLv21rmNx6rHdRFxA7XTbcWGdAPaPGiOcUSsjYhxmbmmOv23rotttB/jhyJiCTCF2nXPgaCe49W+zuqIGAqMZAedTuwD3bY3M9d3mPxnamNsBrMB9TvbW5n5VIfn/xYRX4uIPTJzu79oZECd7o6I44FPAidm5rNdrNZUtyuNiF0iYtf259QG13U64nAQGUzH+EZgdvV8NrDVmYSIeHlEDKue7wEcBTywwyrsvXqOV8f34Z3Aj7v4ED4QdNveLa7Hngis2IH19YcbgdOrUd5TgSfbL/MMRhGxV/uYiog4klrWrt/2q7rQ36PktnNE3Spq1zWWVz/to0H3Bv6tw3pvB35FradxYX/X3Yv2/g21T6B/BtYC/7Fle6mNIP1F9XP/QG5vvW0eZMd4d2AxsLJ6HF3NbwW+UT1/A3BvdYzvBeb0d909aOdWxwu4iNoHboDhwPXV7/hdwH79XXMft/eL1e/rL4BbgNf2d829bO/3gDXAC9Xv7xzg/cD7q+UBXFG9H/eyjb9WGQg/dbT3Qx2O7x3AG3q6L28LKklSoQbU6W5JkpqJIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVD/H4QB6ASTSG02AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#在训练集上观察预测残差的分布，看是否符合模型假设：噪声为0均值的高斯噪声\n",
    "f, ax = plt.subplots(figsize=(7, 5)) \n",
    "f.tight_layout() \n",
    "ax.hist(y_train - y_train_pred_lr,bins=40, label='Residuals Linear', color='b', alpha=.5); \n",
    "ax.set_title(\"Histogram of Residuals\") \n",
    "ax.legend(loc='best');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "残差分布和高斯分布比较匹配，但还是右skew。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 381,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAADQCAYAAADcQn7hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xt8lPWV+PHPmcmQG5dwVyKoVVcrKCDootaKVBe0ioAXtLQrtdV1W7XeULR11e62arFiL/uytWitVlu8siIq3n9VLCjITURaRUECAgFCDAnJZOb8/pgZmCRzzTzPPJPkvF+vvMhMkuc5QObM93q+oqoYY4wbfF4HYIzpvCzBGGNcYwnGGOMaSzDGGNdYgjHGuMYSjDHGNZZgjDGusQRjjHGNJRhjjGuKvA4gG/369dNDDjnE6zCM6fKWLVtWrar9031fh0owhxxyCEuXLvU6DGO6PBHZkMn3WRfJGOMaSzDGGNd4lmBEpERE3hWRlSKyRkTu8CoWY4w7vByDaQTGqWqdiASAt0XkRVVd7GFMxhgHedaC0Yi66MNA9MOK0xjjgaamJleu6+kYjIj4RWQFsA14RVWXJPiey0VkqYgs3b59e/6DNKaTW7VqFaNGjeLxxx93/NqeJhhVDanqCOAg4AQRGZbgex5Q1dGqOrp//7TT7saYDIXDYe69916OP/54qqur6du3r+P3KIhZJFWtAd4EJngcijFdwqZNmzjjjDO4/vrrOfPMM1m1ahXjx493/D5eziL1F5GK6OelwOnAR17FY0xX8t5777FkyRLmzJnDs88+i1u9Ay9nkQ4E/iQifiKJ7glVfd7DeIzp1Hbv3s0777zDmWeeyeTJk1m/fj0DBgxw9Z6eJRhVXQWM9Or+xnQlb731Ft/5znfYvn07GzZsoF+/fq4nFyiQMRhjjDuampr48Y9/zNixY/H7/bz66qv069cvb/fvUJsdjTGZCwaDnHLKKbz77rt873vfY/bs2fTo0SOvMVgLxphOKhAIMHnyZJ5++mnmzJmT9+QClmCM6VS2bt3Kueeey+uvvw7AzJkzmTJlimfxWIIxppOYP38+xxxzDAsXLmTjxo1ehwNYgjGmw9uzZw9XXHEFEydOZNCgQSxbtozp06d7HRZgCcaYDm/u3Lk88MAD3HjjjSxZsoShQ4d6HdI+NotkTAcUCoVYu3Ytw4YNY/r06QwfPpxRo0Z5HVYb1oIxpoP59NNPOfXUUznllFOorq7G5/MVZHIBSzDGdBiqyiOPPMLw4cNZvXo1v/3tb13ZAe0k6yIZ0wEEg0GmTZvGk08+yde//nUeeeQRDj74YK/DSstaMMZ0AIFAgB49enDnnXfy+uuvd4jkAtaCMaZg7d27l1tvvZVLLrmEYcOGMWfOHETE67CyYgnGmAK0evVqpk2bxurVqxkwYADDhg3rcMkFrItkTEEJh8PMnj2b0aNHs3XrVhYsWMCMGTO8DqvdvKxoN1hE3hCRtdFzkX7kVSzGFIo5c+Zw3XXXMWHCBFavXs1ZZ53ldUg58bKL1Axcr6rvi0gPYJmIvKKqH3oYkzGe2LFjB3379uWSSy6hV69eXHjhhR2yS9Sal+cibVHV96OffwmsBSq9iscYL9TW1jJ9+nSOO+44du/eTXFxMVOnTu0UyQUKZAxGRA4hUj7TzkUyXcbbb7/N8OHDefTRR5k+fTplZWVeh+Q4zxOMiHQHngauUdXa1l+3c5FMZ9Pc3MxPfvITTj31VHw+H2+//TZ33HEHgUDA69Ac5/XJjgEiyeUxVX3Gy1iMyRefz8ff//53LrnkElasWMGJJ57odUiu8WyQVyKdzAeBtap6r1dxGJMPqsqDDz7ImWeeSWVlJQsWLKCkpMTrsFznZQvmZOA7wDgRWRH96NhzcsYksG3bNs4991wuu+wy7r//foAukVzA23OR3gY6x1C5yZt5y6uYtXAdm2saGFRRyozxRzJpZP4nH5PF0fr508o28YefzWD37t3cd999XHXVVXmP1Uuiql7HkLHRo0fr0qVLvQ7DeGTe8ipufmY1DcHQvudKA37unHJMXpNMojgEOOmwPry/cfe+5+tWv8aOF2ZzyBFfZf4zTzBs2LC01y2E5JkJEVmmqqPTfZ/tRTIdxqyF61q8qAEagiFmLVyX1xdiojgUWPTJzsjn4RDi81N2xL/S/LVpHDD+2/uSS6qWT3zSqqpp4OZnVgMUbJLJhCUY47lM37k31zQk/Plkz7sl2f00HKJ2ydPU/3MxB0y7G19JdypOvpgv6iJJI1USKZTk6TRLMMZT2bxzD6oopSrBi3tQRanjMaVKeIniaN69lernf0njpg8pO+oUtDmI+CPrWirKIn8mSyLXP7GSUJKhinwnT6dZgjGeyuade8b4IxOOwcwYf2TS689bXsXtz62hpiEIQO+yALedMzRpq2De8ipmPLWSYCjygq+qaWDGUyuB/QnvtKP68+fFkXOHVJU9a95g5yv3A0Lfs6+n/OixLZb676oPcvJdrydMjgAhVYRIN6s1p5NnvlmCMZ7KptsTe4G3mKU5qj+zFq7jmrkr8IsQUqUy2uoAmPHkSoLh/S/dXfVBrpm7gqUbdvI/k45pc4875q/Zl1xigiHljvlr9o2VPL2sav8Xw83ULnmabgMOpd/Z11PUa2DCv09VTUPSJEL0+dZfT5c8OwKbRTKeSvbOXllRyqKZ41L+bKLZnJjSgJ+SgI9d9cGEPyvA7Kkj2rRkDpm5IOn9BBCBsMLejavpNvAwfMVlNNftxF/WC/H5U8Ybu0aqV1xlRanNIhnjlPZ0e2ISda9iGoKhpF+DyIu8dTds3vKqpN8f+5lwMEjN3x6h9r1n6fmv59N77HSKuvdJG2v8NZIlmUySakdjCcZ4KlG3J9N37lwHQKtqGva1oHzRlkkqTds/o3r+PQS3f0b3kWfR66SL2nXfRLfpDN2hRCzBGM9NGlnZrq5ARVkgaRcIoKI0wJ7G5hZjMPEE9nXP0iWX+nXvsH3+LHzF5fQ//zbKDjs+63gT3V9h35hRoXaHcmEJxnQ4sWnkVMmlNODn9olDWbph574Zn9ayGX3sduDhlB15En3GXYa/vCLLiBOLJZdE3aKOtKo3Fc/rwRiTjdjAbrIp35jY9oE3Pmp/kbI96xZRPf8eVJWingPof84Mx5JLTKK/R/zfUdm/NijdGFEhsgRjOpRUA7sx8Tto2zNOE26sp3rBfVTPu5PgzirCe+uyvkamhLaDy6nWBnU0lmBMh5JJwlAi61kAepVmVyVu76a1bHn4avaseZ1eJ07lgG/Pwl/aoz2hZiQ2mxWvULZEOMHGYExBaz0W0as0sG9Vbiq76oOM/OnLGX1vjIaCVM+fBSIM/NZdlBx0dC6hZ6x14sjXloh88Lpk5kMisk1EPvAyDlOYEo1F7GlqJuDLrIxQqkHgeMGaL9BQM+IPMOC8Wxn03d/kLbnA/r1KMTPGH0lpoOWivY46je11F+lhYILHMZgClWgsIhhSmuNWn5cF2v8rrKp8ueIltjz0Q3YvfhKAbgMOxVec3+r+dXubW4zDTBpZyZ1TjqGyohQhMtOU75o3TvG0i6Sqf4seWWJMG0nLIsTNL7feN5Spbk1fsvWFX1G7bjElB4+g+7FntOs6TgiGtc2q4vauDSo0XrdgjEkqkzGHZIvoUtm7YRWf/eEHNHy6nN7jLmPA1J9S1KNfe0JMq3dZgPumjqB3WerB5o44gJuJgk8wdvBa1zRveRW79jS6cm1fSXekez+m/+Ixeh5/LiLuvQxipSHKuqXuLCiRjZ8dca1LKgWfYOzgta4nVpOlPhh27JqNX3xMzaK/ANBt4Fc44N/v5eUt+avsn0kLpSMvqEum4BOM6VzmLa/i5Lte59CZC5K+Y89auC7p2Err+aN0M0oaDrF78ZN88ej11K14iVBD5PDQfJ39HFvjkukUc0ddUJdMynabiFyX6uu5HpgmIn8BxgL9RGQTcJuqPpjLNU1+ZbNnJtPymKne7WP7d+LvF1+xLl7z7m1UL7iXxs8/oOzIk+kz/kpXF80lEvu7JCpLke5nOoN0s0ix/40jgeOB56KPzwH+luvNVfXiXK9hvJNJwohPQL5oxbl4DcEQd8xf0yJJlQZ8SbtHsc2BseteO3dFwtW6GgryxWM3EW6so+83r6V86Li8tVrixVouicpS1Dc1J1yr0xEX1CWTUUU7EXkZOE9Vv4w+7gE8qap5XcNiFe28kayVkqwanV+EsCoVZQHq9iYvl5CtgE+YdcFwgDatAR8QJrKPSLqVIiI0rF9GUZ9KAhUHOHL/bKU7s6lQznlqD6cr2g0BmuIeNwGHtCMu08GkaqUka8rHWimZrqTN1NQTBu9LbK27GmFg78ZVVD8/m14nXkCPkWdR+pVRjt4f0pe8jCXXTEos5FJsq6PINME8CrwrIs8S+fedDDziWlSmYKTa2Ztsz4xbYqUXWt9Tm4PUvPUote8+S1GfQXQ74HBX7h/wC7POH76v+Hf86QOtv56pzrKgLpmMEoyq/kxEXgROiT71XVVd7l5YxkvxXaJk79abaxqYPXUE185dkVXhJgCfRFbjDqooZU9jc8YbEqtqGjjs5hdatCKaqjdGylhuW0/3ERPofdr38XVzfvrZLwmSR+u/eMepn5832UxTlwG1qvorYJOIHOpSTMYBmUwHJ/u5+A2GyQyqKGXSyMp2vaaKi3wMis4EiaSfao4XUm1xz1DtdkJ1O+l/3q30HX+lK8kFIKzaIrnMWriuzdhSbMm/2S+jBCMitwE3ATdHnwoAf3YrKJObXCqiZVLQKX5nb2U7ZjwaguF9se2qD2Y9CNxct5M9H74ZieUro6j8jzmUHf6vWceRSLJU13pmpzPVbHFTpi2YycBEYA+Aqm5m/xS2KTB3zF/T7opo6V4gfpF9sxzzlldR39ScU6zZqv/HO2x56Ep2LPzffYvmnGq1+EWYNmZIRqUSkk0ld6YpZidkmmCaNDKfrQAiUu5eSCYX85ZXJZ29yeTdNd0LJNZViLWSnJ4pSnrfxnqqX/gV25/9OUW9BnDgv8/GX9rTseuXBvz88sLh/M+kYzIqldCZara4KdNZpCdE5PdAhYhcBlwKzHEvLNNeqVopmby7pltxGrtGJl0pp2goyJZHrqN5ZxU9T7yQipMv3newfK4E2kwPZzKzk26KubOcCpCrTGeR7hGRM4BaIqt6/0tVX3E1MtMuqVopmby7xl4EiZbfB/yy7xr5GGtQDSPiQ/wBeh4/iUDfgygZPMyx6+d6kmKyRJTploiuINNB3rtV9RVVnaGqN6jqKyJyt9vBdWbtneVJd01fkuXwFaWBjH+5J42s5PaJQ9vO7sSNxbo91hDctZkv/jyD+o/fBaDHiAmOJhc3uzOd6VSAXGU6BpOo3NeZTgbSlbhx7k3smq33+sD+Q8iykWwa9vonVnLozAXUZ1EbNxuqypcrX2bLH6+meccmCDvXDYuF63YJSpth2i/dbur/BH4AHCYiq+K+1AN4x83AOrNU73Dt/aVPNiYSP+uTjUy2AQT8QkVpgN0NwawXzSW8dv1udrz0Gxr+uZiSg4+l71nXUdTTmUpz+dzj05lOBchVuhbM40R2Tv9f9M/YxyhVneZybJ2WG+9wyX629QKxTGVUrjKklBcX8eld32TRzHGcPfzApOtIMtGwfhkN65fS+7RLGTD1fxxLLpDfLorNMO2XMsGo6m5V/Qz4FbBTVTeo6gYgKCLOrGzqgtxYQ+H0NRO9SBKJJbZ5y6t4ellV1it7w8FG9m76EIDyoacx6Pu/o+cJU1wpY5mvLkpnOhUgV5lOU98PHBf3eE+C50yGEk0F5/oO59Q146dXK8oCFBf5UnZ7fCIcOnNBwlov6TRt/YTq+ffQ/GU1lVc8iL+0Z7tLK5QGfDSkKbGZzy5KZ9/EmKlME4xoXOEYVQ2LSM5HnojIBCKtIz8wR1XvyvWaHYEb2/SzvWaidRrQss7KrvogpQE/FSlOU4wllWySi4ZD1L77LDVv/Rl/WU/6T7ol50VzJQE/fcqLk+7u7qpdFK9lWnDqGeBNIq0WiAz8nqaqk9p9YxE/8A8iM1SbgPeAi1X1w2Q/YwWnnJGs0FFJwOf6ylwNBdn6xH/RuHE1Zf9yEn0mXOnIilwBZk8d0aaEAkSm6G+fONRaFA5yuuDUFcCvgZ8QWQ3xGnB5+8MD4ATgY1VdDyAifwXOBZImGOOMZLNY+ViZK/4AxQf+C92HfYPyYd9wrIxlrPsTapVcfGDJxUMZjaSp6jZVvUhVB6jqQFX9lqpuy/HelcDncY83RZ9rwc5Fcl62g529ywI5zQ6F99ZFim9vjszi9B77Xbofc7qjNXJPO6o/tz+3htajMGEiq5KNN9Ktg7lRVX8hIr8hQTkdVb06h3sn+u1KdI8HgAcg0kXK4X4mKtk6jYrSAI3N4TYtmVy6TXs3rqZ6wb2EvtxBceVXKR7U/nEQkZbHxsZ746PtSceJclmbY3KTrgWzNvrnUmBZgo9cbAIGxz0+CNic4zVNBpKt07h94tB906u50lCQXW8+zNa/3IL4izjg27PoMaL9i78FmH3hiKQtqa64SrYjSNmCUdX50T//5MK93wOOiFbGqwIuAr7lwn1MK6lmnJw6srVu1SvULnmK7sPH03vc9/F1yy1pTRszhEkjK5m1cF3SVbLJjgFJdy60cU+6LtJ8UlQaVdWJ7b2xqjaLyJXAQiLT1A+pqnWWXRKblq6qacAfXbNSWVHK7KkjgEiyuWbuipzuoaqEardR1Gsg3YePp6hPJaUHD8/pmgKcdFgf3vhoO4fOXEBFWYCAT1rsk4qfgk5UiPu2c7Lbh2Wck24W6Z7on1OAA9hfJvNi4LNcb66qLwAv5Hodk1rraenYmpWqmgZmPLkShKRHtWaquW4nO174FU3bPmHQ9+7HX9oj5+QCkZbL08uqWqzNab0HqvV6n/hEGgztr5NrM0n5l66L9P8AROS/VfXrcV+aLyI5n+xo8iNVcSgnDkWr/8ff2fHSb9DgXnqf9j18Jd1zviZENmq+8dH2NrHH9kCtuO3f2vxMLIlYPZbCkOk6mP4i8pW4NSuHAv3dC8u0R7Iqam4NgGooyM6X76du1ct0G3gY/c6+gUC/wel/MEMh1XZtDHVjt7ppn0wTzLXAmyKyPvr4EOA/XInItEuqKmquHZDmKyK090t6jjmfiq9Nc6yMZUxsNivb0gdWj6VwZFoy8yUROQI4KvrUR6qa+1SDcUyyd+3rn1hJSDXtkaeZ0nCI2sVPUXb0qQQqDqD/pJtd2fkcP3Cb7SZOq8dSODItmVkGzACuVNWVwBAROdvVyExW0hWIUpKf+ZOp4K4tfPHYjdS89Sj1ayNDcE4kl9KAn2+PGZKwvEF7Sh9YPZbCkWkX6Y9EFtadGH28CXgSeN6NoAqVk5Xina46n0k3qL0tGFVlz+pX2fnaAyA++p0zg/KjT23n1dpqCIZ446PtSQtwZ1v6oCscKt9RZJpgDlPVqSJyMYCqNoiTG0k6ACcrxae7VrLkkyoppTtuJBd1K19i58L/pXjIMfT75rUU9Rzg+D2cHh+xeiyFIdME0yQipew/eO0woEuNwTg5M5Gu6nyi5LN0w84W60FaJ6X4d22nBnTDwb34AiWUH30aqNJ9xARXxlvAxkc6q0x/W24DXgIGi8hjRMo13OhaVAXIyZmJVNdKlnz+suTzpIO4saNPABbNHEdFaW6zOeFgIztf/T1f/OnaSJLpVkKPkWe5llxsfKTzStuCiXaFPiKymncMkbHCH6lqtcuxFRQnZyaSXcsnkrT1kaxiXPyq3GvmruDmZ1bR2Jy6dGQqTVvXUz3/HoI7NtJj1ETXkkpMpY2PdGppf3uipTLnqeoOVV2gqs93teQCzs5MJCuonarspD/DIa+GYJj2LM5VDbN7ydNseeQ6wo11DLjwp/Q5/XKkqFv2F2sl4Jc2ZyiVBvzcN3UEi2aOs+TSiWU6BrNYRI5X1fdcjaaAOTEzkaig9u6GYNqC2aUBP+eNqmwxBuO4cJj6dYsoO/wEx8pYQiQxzjo/sifJZnW6nkxr8n5I5Ezqz4icKCBEGjfHuhpdKx25Jm+yOrh3TjmGa+euSDqFXBlXkDv+vOhUxZeyseejtyk5+Fj8pT0JN+5BupW1q9KcT2jTcgr4hFkXDLdE0gk5XZPXjonNUaqZo2RjMrHD2RMlp/YcExIv3LiHnS/fz54P36TniVPp/fXv4Csub9e1epcFEtZh6V5SZMmli0s5BiMiJSJyDZFVvBOAqtjha9ED2EyGUs0cpRvfSZScQjnsgt77+QdsfuhK9qz9G72+No2Kr7W/zldlRSk1SUpqJnvedB3pBnn/BIwGVhNpxfzSiZuKyAUiskZEwiKStpnVGaQ6eTHRcvjzRkWqtx06c4GjGxXr1rzB1sdvRnxFHDDtF1ScfDHiS3+CYyKxJOjGSZWmc0jXRTpaVY8BEJEHgXcduu8HRKa9f+/Q9QpeupMX4xfLJeoS5UpVERFKDxlBj9ETqTjl21mXsawoDVBeXJRwoNbpkypN55Auwexr40ZLXDpyU1VdCzh6bEWhi70Q75i/Zt94RXFR4gbk7c+tcSy5qCp1yxdQ/88lDLjgdorKe9PnG5el/JmAX0BpU5Yy2flCtvfHJJMuwQwXkdro5wKURh/HZpGcmctMQUQuJ3rI25AhQ9y+nev2xp2fXNMQbLOfad7yKseO2QjV7aL6xfvYu34ZJYeOQoN7obicyhQbI+NnrbJJGLb3xySSrmRm+zrngIi8SqSOb2s/VtX/y/Q6nelcpGQzSXfMX9OiFeCE+n8uZseLv0aDe+lzxhV0H/lNRGRfskg2ZR6fJCxhmFzlfIB9Mqp6ulvX7qiSzSTtqg8yb3mVY+UttTnIrtcfxN+jH/3OuYFu/fa3/OJbItalMW5zLcGYtlLVbIntys6lvGXjFx8T6DsYX6CYAVP/m6IefVuUsfTJ/lZJ60HlWQvXce3cFZZsjKPc3cmWhIhMFpFNRApYLRCRhV7EkW+pZlViLZcZ44/MuvKchkPUvPNXvnjkOmr//gQAgYoD2tTITbR0JjZjVVXTgLK/DMS85VVZRmFMW54kGFV9VlUPUtViVR2oquO9iCPfJo2sTFpKIbZmZNLIyqwqzwVrvmDr4zPZ/dafKTvqFHqeMDnp9wq0SRzpatMYkwtPEkxXdvbwA9u0UFqvGcn0bOj6j99lyx+voql6I/3OuYH+E2ekPJNIaTuInKw7ZhX4jRMsweTRvOVVPL2sqkULRYDzRrWc4k1WzqG1QO8DKT7oaAZ99zeUHz02oxjiE8e85VVJu2O2Ctc4wQZ5XZCsdm6i7ogCb3y0vcVziRblxTR8+j4Nn7xH729cTqDvYAZecEdWscUnjlkL1yXsjgmpx4uMyZQlGIelKuidTdnNWJKJHUivzU3sevNhvlz2HIG+gwk37sGf5RGtAZ+0SBzJ4lFsDYxxhiUYh7WnLEN8q2Le8qo2LZembZ9SPX8WweqN9Bh1DhWnTscXKM46tu4lkf/uk+96nc01DUlLPmQ6BmRMOpZgHJaqlTJtzBD+vHhjm6/V1Dftm92Z8dRKgqH9L3ptDrLtydtBwwy44A5KvzKq3bHtqg+2aF0lSi62SdE4yRKMw1K1UlqPtcTsaQpxzdwVLarUNdftxF9egRQF6HfuTAJ9BuEv65VzfIk2UfpFCKvaIjvjOJtFcliq4lHppn5jyWXP2rfYMuc/+fK9eQCUHPRVR5JLMmFVPr3rm1aA2zjOEozDUp2lnG7qN9y4h+rnf0n1c3dT1OcgSo8Y40hM6api2JS0cYt1kVzQep/P7c+t2TcblExj1Vq2PzeL0JfV9Dr5YnqddFFWleZKAz76lBcn3Lx46MwFKX7OxlyMeyzBuGje8ipmPLmyReGmZDQcQvxFHDDtboorv5r1vZrDmnT8JNm4kF+kTYkGY5xkXSQXzVq4LmVyCe74nNpl8wEoGTyMQd+/v13JBSAY0qT7h5KNC/3yQjtSxLjLWjAuSrqQTZW6FS+y6/UHkW4llA89DX9J93YX3053P6v/YrxiCcZFFQnOCwrt2cWOF39NwyfvUXLocfQ965qsV+Qmo0QW0SVKHlbS0njBEoxL5i2vom5vc4vntDnIlkeuI7Snht6n/wc9jvum44fLx29NsIRivOZJghGRWcA5QBPwCfBdVa3xIha3xI+/aHMQKQogRQF6j72UQL8hdOt/cM73EEi4WTG2NcESjPGaV4O8rwDDomdb/wO42aM4XBObtWnc8k82//FK9nz4JgDlXz0l5+QSW18ze+qIpOUWrJ6LKQSetGBU9eW4h4uB872Iw0nxJRoqygJoOETt4qeoWfQ4/vLe+Lv3ceQ+sfOqY2YtXJd2A6UxXimEaepLgReTfVFELheRpSKydPv2xHt5vNa6ru22zZ+z9fGbqXnrUcqOPJkDL/0tJUOOdeRerRfFpTvX2hgvudaCyeRcJBH5MdAMPJbsOh3hXKTWJRqavviYpu2f0ffs6yk/emxGJ1gKcNJhffhsR0PSMpYVpYGEs0OxGGwK2hQaz85FEpFLgLOBb6gmqBvQgWyuaSDU8CWNmz+i7LDjKT/qa5QMOSarDYoKfLajgUUzxyU8mzp2dGsiNgVtCpVXs0gTgJuAU1W13osYnFRW/SHr5t5FuKmByv/8I/6S7u3a/RwbmLVWieksvBqD+S3QA3hFRFaIyO88iiMne/fu5brrruPDB2/EX1zOwIvv3LdoLuCXpEeUJBM/MDtpZCWLZo5j9tQRAFw7dwUn3/W6nVdkOhSvZpEO9+K+TmpsbGTMmDGsXLmSH/7wh5zyrWv49d82tmlxJOruJJJoYDZVfV9rzZiOwFbyZklVERGKi4u5+OKL+fnPf85ZZ50FwNST2ubN+O5OVU1DwsVxFaUBbp84tE3SSFXf1xKM6QgswWShqqqKSy+9lFtuuYVTTz2Vm266KaOfS3QOdCZjK9mcQmBMIbIEk6GnnnqKyy+/nMbGRrZs2dLu62Qz45PJKQTGFLJCWGhX0Gpra5k+fToXXHABhx9+OCtWrOCiiy7Ky71tEZ3p6CzBpPHXv/6VRx99lFtvvZUKz9UBAAAGeElEQVRFixZxxBFH5O3eqer7GtMRSEda4zZ69GhdunSp6/cJBoOsXbuWY489lnA4zOrVqxk+fLjr9zWmoxCRZao6Ot33WQumlXXr1nHSSScxduxYdu3ahc/ns+RiTDtZgolSVX7/+99z3HHHsX79ev7whz/Qu3dvr8MypkOzWSQii+bOP/98nn/+ec444wwefvhhBg0a5HVYxnR41oIBiouLGThwIPfddx8vvfSSJRdjHNJlE0x9fT1XX301a9asAWDOnDn86Ec/wufrsv8kxjiuS3aRli1bxrRp01i3bh1HHHEEQ4cmLoNgjMlNl3q7DoVC3HnnnYwZM4a6ujpee+01rrrqKq/DMqbT6lIJ5v777+eWW25hypQprFq1inHjxqX/IWNMu3X6LpKqUl1dTf/+/bnssssYNGgQkydPzqiMpTEmN560YETkv0VkVbTY1Msi4sq0zc6dO7nooos4/vjjqa2tpbi4mClTplhyMSZPvOoizVLVY1V1BPA88F9O3+C1117j2GOP5ZlnnuGKK66gvLzc6VsYY9LwJMGoam3cw3ISH1DYLsFgkBtuuIHTTz+d7t27s3jxYmbOnInfn9vB8saY7Hk2yCsiPxORz4FpONiC8fv9vP/++/zgBz/g/fffZ9SoUU5d2hiTJdd2U2dyLlL0+24GSlT1tiTXuRy4HGDIkCGjNmzYkPbeTU1NdOvWrV1xG2PSy3Q3teflGkTkYGCBqg5L9735KtdgjEmtoMs1iEh81aaJwEdexGGMcZdX62DuEpEjgTCwAbjCoziMMS7y6lyk87y4rzEmv7rUVgFjTH5ZgjHGuMbzWaRsiMh2ImM26fQDql0OJ1OFEkuhxAGFE4vF0VamsRysqv3TfVOHSjCZEpGlmUyh5UOhxFIocUDhxGJxtOV0LNZFMsa4xhKMMcY1nTXBPOB1AHEKJZZCiQMKJxaLoy1HY+mUYzDGmMLQWVswxpgCYAnGGOOaTptg8lWWM4M4ZonIR9FYnhWRCi/iiMZygYisEZGwiOR9WlREJojIOhH5WERm5vv+cXE8JCLbROQDr2KIxjFYRN4QkbXR/5cfeRRHiYi8KyIro3Hc4djFVbVTfgA94z6/GvidR3H8G1AU/fxu4G4P/02+ChwJvAmMzvO9/cAnwFeAbsBK4GiP/h2+DhwHfODV/0U0jgOB46Kf9wD+4cW/CSBA9+jnAWAJMMaJa3faFoy6WJYzyzheVtXm6MPFwEFexBGNZa2qrvPo9icAH6vqelVtAv4KnOtFIKr6N2CnF/duFccWVX0/+vmXwFqg0oM4VFXrog8D0Q9HXi+dNsGAe2U5c3Ap8KLXQXikEvg87vEmPHgxFSoROQQYSaT14MX9/SKyAtgGvKKqjsTRoROMiLwqIh8k+DgXQFV/rKqDgceAK72KI/o9Pwaao7G4JpNYPJLorBhbIwGISHfgaeCaVi3vvFHVkEZO+TgIOEFE0laYzESHPnhNVU/P8FsfBxYACev+uh2HiFwCnA18Q6MdXbdk8W+Sb5uAwXGPDwI2exRLwRCRAJHk8piqPuN1PKpaIyJvAhOAnAfBO3QLJpVCKcspIhOAm4CJqlrvRQwF4j3gCBE5VES6ARcBz3kck6ckcgLgg8BaVb3Xwzj6x2Y3RaQUOB2HXi+ddiWviDxNZMZkX1lOVa3yII6PgWJgR/SpxarqSYlQEZkM/AboD9QAK1R1fB7vfxZwH5EZpYdU9Wf5unerOP4CjCVSmmArcJuqPuhBHF8D3gJWE/k9BbhFVV/IcxzHAn8i8v/iA55Q1Z86cu3OmmCMMd7rtF0kY4z3LMEYY1xjCcYY4xpLMMYY11iCMca4pkMvtDOFQUT6Aq9FHx4AhIDt0ccnRPcemS7IpqmNo0TkdqBOVe9p9bwQ+X0LJ/xB0ylZF8m4RkQOj+6D+h3wPjBYRGrivn6RiMyJfj5QRJ4RkaXR2iRjvIrbOMcSjHHb0cCDqjoSSLWS+tfALzRyJs+FwJx8BGfcZWMwxm2fqOp7GXzf6cCRkZ4UAL1FpFRVG9wLzbjNEoxx2564z8O0LNtQEve5YAPCnY51kUzeRAd4d4nIESLiAybHfflV4IexByIyIt/xGedZgjH5dhPwEpFp7U1xz/8QODlaHP1D4DIvgjPOsmlqY4xrrAVjjHGNJRhjjGsswRhjXGMJxhjjGkswxhjXWIIxxrjGEowxxjX/H1EGfO/tyJDsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 288x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#还可以观察预测值与真值的散点图\n",
    "plt.figure(figsize=(4, 3))\n",
    "plt.scatter(y_train, y_train_pred_lr)\n",
    "plt.plot([-3, 3], [-3, 3], '--k')   \n",
    "plt.axis('tight')\n",
    "plt.xlabel('True')\n",
    "plt.ylabel('Predicted')\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**2. 正则化的线性回归（L2正则 --> 岭回归）-- 使用 5-fold RidgeCV**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 382,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "alpha is: 1\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef_lr</th>\n",
       "      <th>coef_ridge</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>[81468554938586.86]</td>\n",
       "      <td>[0.4484156335610655]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>[81468554938586.69]</td>\n",
       "      <td>[0.2111351597600545]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>[81468554938585.81]</td>\n",
       "      <td>[-0.6595507933213497]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>workingday</td>\n",
       "      <td>[27391236788852.863]</td>\n",
       "      <td>[0.10166892293776165]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>holiday</td>\n",
       "      <td>[27391236788852.582]</td>\n",
       "      <td>[-0.16688988956593867]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>[26330226186121.223]</td>\n",
       "      <td>[0.12196409869969428]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>[26330226186121.03]</td>\n",
       "      <td>[-0.056743132071508606]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>temp</td>\n",
       "      <td>[1.6257144485015205]</td>\n",
       "      <td>[1.0063601672600992]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>yr</td>\n",
       "      <td>[1.0310554943013157]</td>\n",
       "      <td>[1.0405499131158658]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>atemp</td>\n",
       "      <td>[0.3700555230824113]</td>\n",
       "      <td>[0.8083373724741407]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>[-0.7869353645955468]</td>\n",
       "      <td>[-0.6813027932711112]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>hum</td>\n",
       "      <td>[-0.9067522720326273]</td>\n",
       "      <td>[-0.7165574285528961]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>[-1061010602731.6381]</td>\n",
       "      <td>[0.04633234950567558]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>[-1061010602731.6528]</td>\n",
       "      <td>[0.0210084982163389]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>[-1061010602731.6815]</td>\n",
       "      <td>[-0.00460898602983515]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>[-1061010602731.7129]</td>\n",
       "      <td>[-0.07503869094688177]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>[-1061010602731.742]</td>\n",
       "      <td>[-0.052914137373443464]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_4</td>\n",
       "      <td>[-11647555030435.684]</td>\n",
       "      <td>[0.40432968311143325]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_3</td>\n",
       "      <td>[-11647555030436.236]</td>\n",
       "      <td>[-0.015901170727774524]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_2</td>\n",
       "      <td>[-11647555030436.273]</td>\n",
       "      <td>[-0.012402471514923366]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>season_1</td>\n",
       "      <td>[-11647555030436.523]</td>\n",
       "      <td>[-0.37602604086873226]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>[-20261068421518.26]</td>\n",
       "      <td>[0.2488524783926767]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>[-20261068421518.312]</td>\n",
       "      <td>[0.32979056524904443]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>[-20261068421518.44]</td>\n",
       "      <td>[0.11437502739429317]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>[-20261068421518.508]</td>\n",
       "      <td>[0.07414051960277476]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>[-20261068421518.527]</td>\n",
       "      <td>[0.10825202290351885]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>[-20261068421518.547]</td>\n",
       "      <td>[0.12979084687281187]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>[-20261068421518.613]</td>\n",
       "      <td>[0.06091920668198116]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>[-20261068421518.754]</td>\n",
       "      <td>[-0.10168596947290354]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>[-20261068421518.816]</td>\n",
       "      <td>[-0.25376846934242697]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>[-20261068421518.84]</td>\n",
       "      <td>[-0.19392861234171235]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>[-20261068421518.9]</td>\n",
       "      <td>[-0.25401852960555443]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>[-20261068421518.92]</td>\n",
       "      <td>[-0.26271908633456986]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns                coef_lr               coef_ridge\n",
       "16  weathersit_1    [81468554938586.86]     [0.4484156335610655]\n",
       "17  weathersit_2    [81468554938586.69]     [0.2111351597600545]\n",
       "18  weathersit_3    [81468554938585.81]    [-0.6595507933213497]\n",
       "31    workingday   [27391236788852.863]    [0.10166892293776165]\n",
       "30       holiday   [27391236788852.582]   [-0.16688988956593867]\n",
       "25     weekday_6   [26330226186121.223]    [0.12196409869969428]\n",
       "19     weekday_0    [26330226186121.03]  [-0.056743132071508606]\n",
       "26          temp   [1.6257144485015205]     [1.0063601672600992]\n",
       "32            yr   [1.0310554943013157]     [1.0405499131158658]\n",
       "27         atemp   [0.3700555230824113]     [0.8083373724741407]\n",
       "29     windspeed  [-0.7869353645955468]    [-0.6813027932711112]\n",
       "28           hum  [-0.9067522720326273]    [-0.7165574285528961]\n",
       "24     weekday_5  [-1061010602731.6381]    [0.04633234950567558]\n",
       "23     weekday_4  [-1061010602731.6528]     [0.0210084982163389]\n",
       "22     weekday_3  [-1061010602731.6815]   [-0.00460898602983515]\n",
       "20     weekday_1  [-1061010602731.7129]   [-0.07503869094688177]\n",
       "21     weekday_2   [-1061010602731.742]  [-0.052914137373443464]\n",
       "3       season_4  [-11647555030435.684]    [0.40432968311143325]\n",
       "2       season_3  [-11647555030436.236]  [-0.015901170727774524]\n",
       "1       season_2  [-11647555030436.273]  [-0.012402471514923366]\n",
       "0       season_1  [-11647555030436.523]   [-0.37602604086873226]\n",
       "8         mnth_5   [-20261068421518.26]     [0.2488524783926767]\n",
       "12        mnth_9  [-20261068421518.312]    [0.32979056524904443]\n",
       "7         mnth_4   [-20261068421518.44]    [0.11437502739429317]\n",
       "6         mnth_3  [-20261068421518.508]    [0.07414051960277476]\n",
       "11        mnth_8  [-20261068421518.527]    [0.10825202290351885]\n",
       "9         mnth_6  [-20261068421518.547]    [0.12979084687281187]\n",
       "13       mnth_10  [-20261068421518.613]    [0.06091920668198116]\n",
       "10        mnth_7  [-20261068421518.754]   [-0.10168596947290354]\n",
       "4         mnth_1  [-20261068421518.816]   [-0.25376846934242697]\n",
       "5         mnth_2   [-20261068421518.84]   [-0.19392861234171235]\n",
       "14       mnth_11    [-20261068421518.9]   [-0.25401852960555443]\n",
       "15       mnth_12   [-20261068421518.92]   [-0.26271908633456986]"
      ]
     },
     "execution_count": 382,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#岭回归／L2正则\n",
    "#class sklearn.linear_model.RidgeCV(alphas=(0.1, 1.0, 10.0), fit_intercept=True, \n",
    "#                                  normalize=False, scoring=None, cv=None, gcv_mode=None, \n",
    "#                                  store_cv_values=False)\n",
    "\n",
    "from sklearn.linear_model import  RidgeCV\n",
    "\n",
    "#设置超参数（正则参数）范围\n",
    "alphas = [ 0.01, 0.1, 1, 10, 100]\n",
    "\n",
    "#生成一个 5-fold RidgeCV实例\n",
    "ridge = RidgeCV(alphas=alphas, cv = 5)\n",
    "\n",
    "#模型训练\n",
    "ridge.fit(X_train, y_train)  \n",
    "\n",
    "#预测\n",
    "y_test_pred_ridge = ridge.predict(X_test)\n",
    "y_train_pred_ridge = ridge.predict(X_train)\n",
    "\n",
    "#输出最佳正则超参数\n",
    "print (\"alpha is:\", ridge.alpha_)\n",
    "\n",
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(columns), \"coef_lr\":list((lr.coef_.T)), \"coef_ridge\":list((ridge.coef_.T))})\n",
    "fs.sort_values(by=['coef_lr'],ascending=False)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**模型评价**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 383,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of RidgeCV on test is 0.39812996248650895\n",
      "The RMSE of RidgeCV on train is 0.3910932883319325\n"
     ]
    }
   ],
   "source": [
    "# 评估，使用 RMSE 评价模型在测试集和训练集上的性能\n",
    "rmse_test = np.sqrt(mean_squared_error(y_test, y_test_pred_ridge))\n",
    "print(\"The RMSE of RidgeCV on test is {}\".format(rmse_test))\n",
    "rmse_train = np.sqrt(mean_squared_error(y_train, y_train_pred_ridge))\n",
    "print(\"The RMSE of RidgeCV on train is {}\".format(rmse_train))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 384,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAekAAAFsCAYAAADos0H8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHxZJREFUeJzt3XuUHWWZ7/HvQ9JMkEDCJUBIxAACgkBCaJDAApEoMKgkrlGDIAYN4HiZ5RyFMXjtcfQMHjzOwctwjEaJCgJyBkFwHDgIg+ORSwJBwChBCJrQJk3GAAlySXjOH7s60yTd6d3du7vf7v5+1uq1d1W9u+p5d6Xz229Vde3ITCRJUnm2G+wCJElS5wxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0RryIeCgiThzsOgZTRLwtIv4QEesj4ogB3O76iNivi2XnRMR/NGg7KyLijY1YlzSQDGkNa53957zlf/6Z+drMvL2b9UyJiIyI0f1U6mD7EvDhzBybmfdtubDq+4YqVFdFxJcjYlRfN1pt79G+rkcargxpqQAFhP+rgIe6aTM1M8cCrwfmAO/r96qkEc6Q1ojXcbQdEUdHxOKIeDoiVkfEl6tmd1SP66rR5IyI2C4iPhURj0fEmoj4bkSM67De91TL1kbEp7fYTktEXBsR34+Ip4Fzqm3/MiLWRURrRHwtIrbvsL6MiA9GxPKIeCYi/iEi9q9e83REXNOx/RZ97LTWiPiLiFgPjALuj4jfdfd+ZeYjwC+AaR3WPy4iFlZ1r4qIz7ePtCPi1RHx7xHxVEQ8GRFXb9GnV1fPd4uIG6q+3A3s36HdVkcyIuL2iDi3er5/RPyseq+fjIgrImJ8F+9FV/tYKo4hLb3cpcClmbkztZC4ppp/QvU4vjpE+0vgnOrnDcB+wFjgawARcQjwz8BZwERgHDBpi23NAq4FxgNXAJuA/wbsDswAZgIf3OI1pwJHAscAfwcsqLbxSuBQ4F1d9KvTWjPz+Wp0DLWR8v6dv/y/RMRrgOOBRzrMXgRsBF4NHAGcDJxbLfsH4GZgF2Ay8NUuVv114Dlq79f76NlIPYB/BPYGDqb2frR00barfSwVx5DWSPCjanS6LiLWUQvPrrwIvDoids/M9Zl55zbangV8OTMfzcz1wEXAGdVo7+3AjzPzPzLzBeAzwJY3yv9lZv4oM1/KzD9n5pLMvDMzN2bmCuAb1A4td/TFzHw6Mx8CHgRurrb/FPCv1AKyp7XW696I2AAsA26neh8jYk/gL4G/zcwNmbkG+CfgjOp1L1I7nL53Zj6XmVtdDFaNuv8K+Ey1jgepBX9dMvORzLyl+tDRBnyZrd+7dj3Zx9KgMqQ1EszOzPHtP2w9Ou1oHnAg8JuIuCci3rKNtnsDj3eYfhwYDexZLftD+4LMfBZYu8Xr/9BxIiIOjIgbI+KP1SHw/05tVN3R6g7P/9zJ9Fg6t61a6zW9Wv8c4HXAjtX8VwFNQGuHD0LfAPaolv8dtZHu3VG7kr6zEfKEqp6O78njnbTrVETsERFXVYfanwa+z9bvXbue7GNpUBnSUgeZuTwz30UtYL4IXBsRO7L1KBjgCWoB1W4faod8VwOt1A7tAhAROwC7bbm5LaYvA34DHFAdiv0EtXBrhG3VWresuQb4JbWjA1AL1ueB3Tt8GNo5M19bveaPmXleZu4NvB/45/bz0B20VfW8cosa222oHl/RYd5eHZ7/I7X38/DqvXs3Xbx329jHUnEMaamDiHh3REzIzJeAddXsTdRC5CVq53Pb/QD4bxGxb0SMpTbyvTozN1I71/zWiDi2upjr7+k+cHcCngbWV+d9P9Cwjm271t64GDg/IvbKzFZq55z/Z0TsXF2ktn9EvB4gIt4REe0fWP5ELUw3dVxZZm4C/gVoiYhXVOf053ZY3gasAt4dEaOq0XjH8+c7AeupXdg3Cbiwq8K3sY+l4hjS0sudCjxUXfF8KXBGdR71WeALwC+qQ7rHAN8Gvkftyu/HqF309DcA1TnjvwGuojaqfgZYQ23E2ZULgDOrtt8Ert5G257qstbeyMwHgH/nv8LwPcD2wK+pBfG11C4AAzgKuKt6T28APpKZj3Wy2g9TO5z+R+By4DtbLD+v2t5a4LXA/+uw7O+pHY5/CriJWuB3pdN9vO0eS4MjMjs7iiepkarR6zpqh7I7CyhJ2oojaamfRMRbq0O3O1K7o9cDwIrBrUrSUGJIS/1nFrULtp4ADqB2WNVDV5Lq5uFuSZIK5UhakqRCDehN/XffffecMmXKQG5SkqSiLFmy5MnMnFBP2wEN6SlTprB48eKB3KQkSUWJiLrvpufhbkmSCmVIS5JUKENakqRCDeg5aUlS91588UVWrlzJc895t9KhbMyYMUyePJmmpqZer8OQlqTCrFy5kp122okpU6YQ0agvQtNAykzWrl3LypUr2XfffXu9Hg93S1JhnnvuOXbbbTcDegiLCHbbbbc+Hw0xpCWpQAb00NeIfWhIS5JUKM9JS1LhWloGfn2jRo3isMMOY+PGjey7775873vfY/z48T3e1rnnnstHP/pRDjnkkJfNv/zyy1m8eDFf+9rXerxOgLFjx7J+/fq62p544om0trYyZswYtt9+e775zW8ybdo0AE477TSuvPLKrfrW0tLC2LFjueCCC3pVX6M4kpYkbWWHHXZg6dKlPPjgg+y66658/etf79V6vvWtb20V0IPhiiuu4P777+eDH/wgF1544eb5P/nJT3r14WOgGNKSpG2aMWMGq1at2jx9ySWXcNRRR3H44Yfz2c9+FoANGzbw5je/malTp3LooYdy9dVXA7VRbPvtoL/zne9w4IEH8vrXv55f/OIXm9d3zjnncO21126eHjt2LADr169n5syZTJ8+ncMOO4zrr79+q9paW1s54YQTmDZtGoceeig///nPe9SXKVOm8OSTTwLwhS98gYMOOog3vvGN/Pa3v93c5p577uHwww9nxowZXHjhhRx66KEAbNq0iQsvvHDze/GNb3yjjnezZzzcLUnq0qZNm7j11luZN28eADfffDPLly/n7rvvJjM5/fTTueOOO2hra2PvvffmpptuAuCpp5562XpaW1v57Gc/y5IlSxg3bhxveMMbOOKII7a57TFjxnDdddex88478+STT3LMMcdw+umnv+yCrCuvvJJTTjmFT37yk2zatIlnn312m+v86U9/yuzZs7eav2TJEq666iruu+8+Nm7cyPTp0znyyCMBeO9738uCBQs49thjmT9//ubXLFy4kHHjxnHPPffw/PPPc9xxx3HyySf36U+utmRIS5K28uc//5lp06axYsUKjjzySN70pjcBtZC++eabNwfs+vXrWb58OccffzwXXHABH//4x3nLW97C8ccf/7L13XXXXZx44olMmFD78qc5c+bw8MMPb7OGzOQTn/gEd9xxB9tttx2rVq1i9erV7LXXXpvbHHXUUbzvfe/jxRdfZPbs2ZvPNW/prLPOYsOGDWzatIl77713q+U///nPedvb3sYrXvEKAE4//XQA1q1bxzPPPMOxxx4LwJlnnsmNN964+b341a9+tfkowFNPPcXy5csbGtIe7pYkbaX9nPTjjz/OCy+8sPmcdGZy0UUXsXTpUpYuXcojjzzCvHnzOPDAA1myZAmHHXYYF110EZ/73Oe2WmdXf5I0evRoXnrppc3rf+GFF4DaeeS2tjaWLFnC0qVL2XPPPbf6u+MTTjiBO+64g0mTJnH22Wfz3e9+t9NtXHHFFTz22GOceeaZfOhDH+q0TWf1ZWYX71Bt2Ve/+tXN78Vjjz3GySef3GX73nAkLak49V7N3OirnrW1cePG8ZWvfIVZs2bxgQ98gFNOOYVPf/rTnHXWWYwdO5ZVq1bR1NTExo0b2XXXXXn3u9/N2LFjufzyy1+2nte97nV85CMfYe3atey888788Ic/ZOrUqUDtvPCSJUt45zvfyfXXX8+LL74I1Eame+yxB01NTdx22208/vjW3/D4+OOPM2nSJM477zw2bNjAvffey3ve855O+9LU1MTnP/959t9/f5YtW8bBBx+8edkJJ5zAOeecw/z589m4cSM//vGPef/7388uu+zCTjvtxJ133skxxxzDVVddtfk1p5xyCpdddhknnXQSTU1NPPzww0yaNIkdd9yxr2/7Zt2GdEQcBFzdYdZ+wGeA71bzpwArgHdm5p8aVpkkCRj8DyNHHHEEU6dO5aqrruLss89m2bJlzJgxA6hd5PX973+fRx55hAsvvJDtttuOpqYmLrvsspetY+LEibS0tDBjxgwmTpzI9OnT2bRpEwDnnXces2bN4uijj2bmzJmbQ+6ss87irW99K83NzUybNo3XvOY1W9V2++23c8kll9DU1MTYsWO7HEm322GHHfjYxz7Gl770JRYuXLh5/vTp05kzZw7Tpk3jVa961csO1y9cuJDzzjuPHXfckRNPPJFx48YBtT8vW7FiBdOnTyczmTBhAj/60Y968Q53LbY1lN+qccQoYBXwOuBDwH9m5sURMR/YJTM/vq3XNzc3Z/tVfpLUlZE+kt5ylKfBtX79+s1XnF988cW0trZy6aWX1vXazvZlRCzJzOZ6Xt/Tc9Izgd9l5uPALGBRNX8RsPXlcpIkDXE33XTTy/7E61Of+tSAbbun56TPAH5QPd8zM1sBMrM1Ivbo7AURcT5wPsA+++zT2zolSRoUc+bMYc6cOYOy7bpH0hGxPXA68MOebCAzF2Rmc2Y2t196L0natp6cilSZGrEPe3K4+y+BezNzdTW9OiImAlSPa/pcjSSJMWPGsHbtWoN6CGv/PukxY8b0aT09Odz9Lv7rUDfADcBc4OLqcev7tUmSemzy5MmsXLmStra2wS5FfTBmzBgmT57cp3XUFdIR8QrgTcD7O8y+GLgmIuYBvwfe0adKJA1rw/VK7P7Q1NTU0LtWaeiqK6Qz81lgty3mraV2tbckSeoH3hZUkqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhRo92AVIUm+1tDS2nVQaR9KSJBXKkJYkqVCGtCRJhTKkJUkqVF0hHRHjI+LaiPhNRCyLiBkRsWtE3BIRy6vHXfq7WEmSRpJ6R9KXAj/NzNcAU4FlwHzg1sw8ALi1mpYkSQ3SbUhHxM7ACcBCgMx8ITPXAbOARVWzRcDs/ipSkqSRqJ6R9H5AG/CdiLgvIr4VETsCe2ZmK0D1uEdnL46I8yNicUQsbmtra1jhkiQNd/WE9GhgOnBZZh4BbKAHh7Yzc0FmNmdm84QJE3pZpiRJI089Ib0SWJmZd1XT11IL7dURMRGgelzTPyVKkjQydRvSmflH4A8RcVA1aybwa+AGYG41by5wfb9UKEnSCFXvvbv/BrgiIrYHHgXeSy3gr4mIecDvgXf0T4mSJI1MdYV0Zi4FmjtZNLOx5UiSpHbecUySpEIZ0pIkFcqQliSpUPVeOCZJnWppGewKpOHLkbQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCjB7sASWVqaRnsCiQ5kpYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKVde3YEXECuAZYBOwMTObI2JX4GpgCrACeGdm/ql/ypQkaeTpyUj6DZk5LTObq+n5wK2ZeQBwazUtSZIapC+Hu2cBi6rni4DZfS9HkiS1qzekE7g5IpZExPnVvD0zsxWgetyjsxdGxPkRsTgiFre1tfW9YkmSRoi6zkkDx2XmExGxB3BLRPym3g1k5gJgAUBzc3P2okZJkkakukbSmflE9bgGuA44GlgdERMBqsc1/VWkJEkjUbchHRE7RsRO7c+Bk4EHgRuAuVWzucD1/VWkJEkjUT2Hu/cErouI9vZXZuZPI+Ie4JqImAf8HnhH/5UpSdLI021IZ+ajwNRO5q8FZvZHUZIkyTuOSZJULENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCjB7sASQOrpWWwK5BUL0fSkiQVypCWJKlQhrQkSYUypCVJKlTdIR0RoyLivoi4sZreNyLuiojlEXF1RGzff2VKkjTy9GQk/RFgWYfpLwL/lJkHAH8C5jWyMEmSRrq6QjoiJgNvBr5VTQdwEnBt1WQRMLs/CpQkaaSqdyT9v4C/A16qpncD1mXmxmp6JTCpsxdGxPkRsTgiFre1tfWpWEmSRpJuQzoi3gKsycwlHWd30jQ7e31mLsjM5sxsnjBhQi/LlCRp5KnnjmPHAadHxGnAGGBnaiPr8RExuhpNTwae6L8yJUkaebodSWfmRZk5OTOnAGcAP8vMs4DbgLdXzeYC1/dblZIkjUB9+TvpjwMfjYhHqJ2jXtiYkiRJEvTwCzYy83bg9ur5o8DRjS9Jkhqr3i8V8ctHVBrvOCZJUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBVq9GAXIEmlaGnpn7ZSbzmSliSpUIa0JEmFMqQlSSqUIS1JUqG6DemIGBMRd0fE/RHxUET8fTV/34i4KyKWR8TVEbF9/5crSdLIUc9I+nngpMycCkwDTo2IY4AvAv+UmQcAfwLm9V+ZkiSNPN2GdNasryabqp8ETgKureYvAmb3S4WSJI1QdZ2TjohREbEUWAPcAvwOWJeZG6smK4FJXbz2/IhYHBGL29raGlGzJEkjQl0hnZmbMnMaMBk4Gji4s2ZdvHZBZjZnZvOECRN6X6kkSSNMj67uzsx1wO3AMcD4iGi/Y9lk4InGliZJ0shWz9XdEyJifPV8B+CNwDLgNuDtVbO5wPX9VaQkSSNRPffunggsiohR1EL9msy8MSJ+DVwVEZ8H7gMW9mOdkiSNON2GdGb+Cjiik/mPUjs/LUmS+oF3HJMkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKlQ9X7AhqXAtLYNdgaT+4EhakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFGj3YBUjSUNTS0th2Ume6HUlHxCsj4raIWBYRD0XER6r5u0bELRGxvHrcpf/LlSRp5KjncPdG4GOZeTBwDPChiDgEmA/cmpkHALdW05IkqUG6DenMbM3Me6vnzwDLgEnALGBR1WwRMLu/ipQkaSTq0YVjETEFOAK4C9gzM1uhFuTAHo0uTpKkkazukI6IscD/Af42M5/uwevOj4jFEbG4ra2tNzVKkjQi1RXSEdFELaCvyMx/qWavjoiJ1fKJwJrOXpuZCzKzOTObJ0yY0IiaJUkaEeq5ujuAhcCyzPxyh0U3AHOr53OB6xtfniRJI1c9fyd9HHA28EBELK3mfQK4GLgmIuYBvwfe0T8lSpI0MnUb0pn5H0B0sXhmY8uRJEntvC2oJEmFMqQlSSqUIS1JUqH8gg2pYH45gzSyOZKWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBVq9GAXIEnDWUtLY9tpZHEkLUlSoQxpSZIKZUhLklQoz0lLg8Dzj5Lq4UhakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQnUb0hHx7YhYExEPdpi3a0TcEhHLq8dd+rdMSZJGnnpG0pcDp24xbz5wa2YeANxaTUuSpAbqNqQz8w7gP7eYPQtYVD1fBMxucF2SJI14vf2CjT0zsxUgM1sjYo+uGkbE+cD5APvss08vNycNDX5xhqRG6vcLxzJzQWY2Z2bzhAkT+ntzkiQNG70N6dURMRGgelzTuJIkSRL0PqRvAOZWz+cC1zemHEmS1K6eP8H6AfBL4KCIWBkR84CLgTdFxHLgTdW0JElqoG4vHMvMd3WxaGaDa5EkSR14xzFJkgplSEuSVChDWpKkQvX2ZibSiOENSjQQevLvzH+TI4cjaUmSCmVIS5JUKENakqRCeU5aI5bn9SSVzpG0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgrlF2xoyPALMSSNNI6kJUkqlCEtSVKhDGlJkgrlOelhrN5zuP1xrncwty0Nd43+vfH3sFyOpCVJKpQhLUlSoQxpSZIK5Tlp9YjnwqThx+tXyuVIWpKkQhnSkiQVypCWJKlQhrQkSYUa0heO9eRCgsG64Kk/ahypF1BIUiMNhf9z+zSSjohTI+K3EfFIRMxvVFGSJKkPIR0Ro4CvA38JHAK8KyIOaVRhkiSNdH0ZSR8NPJKZj2bmC8BVwKzGlCVJkiIze/fCiLcDp2bmudX02cDrMvPDW7Q7Hzi/mjwI+G3vy63L7sCT/byNEtjP4cV+Di/2c3hpdD9flZkT6mnYlwvHopN5WyV+Zi4AFvRhOz0SEYszs3mgtjdY7OfwYj+HF/s5vAxmP/tyuHsl8MoO05OBJ/pWjiRJateXkL4HOCAi9o2I7YEzgBsaU5YkSer14e7M3BgRHwb+DRgFfDszH2pYZb03YIfWB5n9HF7s5/BiP4eXQetnry8ckyRJ/cvbgkqSVChDWpKkQg35kI6ISyLiNxHxq4i4LiLGd9FuSN/CNCLeEREPRcRLEdHlnwJExIqIeCAilkbE4oGssRF60M+hvj93jYhbImJ59bhLF+02VftyaUQMmQszu9s/EfEXEXF1tfyuiJgy8FX2XR39PCci2jrsw3MHo86+iIhvR8SaiHiwi+UREV+p3oNfRcT0ga6xEero54kR8VSHffmZASksM4f0D3AyMLp6/kXgi520GQX8DtgP2B64HzhksGvvYT8PpnYzmNuB5m20WwHsPtj19mc/h8n+/B/A/Or5/M7+3VbL1g92rb3oW7f7B/gg8L+r52cAVw923f3Uz3OArw12rX3s5wnAdODBLpafBvwrtXtnHAPcNdg191M/TwRuHOi6hvxIOjNvzsyN1eSd1P5ee0tD/hammbksM/v7bm2Drs5+Dvn9Sa3eRdXzRcDsQayl0erZPx37fy0wMyI6u0FSyYbDv8NuZeYdwH9uo8ks4LtZcycwPiImDkx1jVNHPwfFkA/pLbyP2ie6LU0C/tBhemU1bzhK4OaIWFLdknU4Gg77c8/MbAWoHvfoot2YiFgcEXdGxFAJ8nr2z+Y21Yfsp4DdBqS6xqn33+FfVYeBr42IV3ayfKgbDr+P9ZoREfdHxL9GxGsHYoND4vukI+L/Ant1suiTmXl91eaTwEbgis5W0cm84v72rJ5+1uG4zHwiIvYAbomI31SfEIvRgH4O+f3Zg9XsU+3P/YCfRcQDmfm7xlTYb+rZP0NiH3ajnj78GPhBZj4fEX9N7ejBSf1e2cAaDvuyHvdSu+f2+og4DfgRcEB/b3RIhHRmvnFbyyNiLvAWYGZWJw+2MCRuYdpdP+tcxxPV45qIuI7aIbmiQroB/Rzy+zMiVkfExMxsrQ4NruliHe3789GIuB04gtp50JLVs3/a26yMiNHAOAo81NiNbvuZmWs7TH6T2nUzw82Q+H3sq8x8usPzn0TEP0fE7pnZr18wMuQPd0fEqcDHgdMz89kumo2IW5hGxI4RsVP7c2oX1XV6peIQNxz25w3A3Or5XGCrIwgRsUtE/EX1fHfgOODXA1Zh79Wzfzr2/+3Az7r4gF2ybvu5xbnZ04FlA1jfQLkBeE91lfcxwFPtp3KGk4jYq/26iYg4mlp+rt32qxpgsK+o6+sP8Ai18yFLq5/2K0b3Bn7Sod1pwMPURiGfHOy6e9HPt1H7xPo8sBr4ty37Se0q0/urn4eGaz+Hyf7cDbgVWF497lrNbwa+VT0/Fnig2p8PAPMGu+4e9G+r/QN8jtqHaYAxwA+r39+7gf0Gu+Z+6uc/Vr+L9wO3Aa8Z7Jp70ccfAK3Ai9Xv5jzgr4G/rpYH8PXqPXiAbfz1Sck/dfTzwx325Z3AsQNRl7cFlSSpUEP+cLckScOVIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVD/HyfFXq1MvfjkAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#在训练集上观察预测残差的分布，看是否符合模型假设：噪声为0均值的高斯噪声\n",
    "f, ax = plt.subplots(figsize=(7, 5)) \n",
    "f.tight_layout() \n",
    "ax.hist(y_train - y_train_pred_ridge,bins=40, label='Residuals Ridge', color='b', alpha=.5); \n",
    "ax.set_title(\"Histogram of Residuals\") \n",
    "ax.legend(loc='best');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 385,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAADQCAYAAADcQn7hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VPXV+PHPmckQErawisR9qQvIrqLUjaqgVARc0Af7SK36+LRqXUrFXdunP6m40Na+rBZaq9WqdaFCqiiirUtRQTYRaV2RiEKAEEO2yeT8/pgZHMIsdzJz584k5/165WUmmdx7BO7Jdz1fUVWMMcYNPq8DMMa0X5ZgjDGusQRjjHGNJRhjjGsswRhjXGMJxhjjGkswxhjXWIIxxrjGEowxxjVFXgeQjj59+uh+++3ndRjGdHjLli2rUtW+qd5XUAlmv/32Y+nSpV6HYUyHJyKfOXmfdZGMMa6xBGOMcY1nCUZEOovI2yKyUkTWiMjtXsVijHGHl2MwjcAYVa0VkQDwuog8r6pLPIzJGJNFnrVgNKw28jIQ+bDiNMZ4oKmpyZXrejoGIyJ+EVkBbAJeUtW34rznUhFZKiJLN2/enPsgjWnnVq1axYgRI3jssceyfm1PE4yqhlR1KLAXcJSIDIrzngdVdaSqjuzbN+W0uzHGoZaWFu655x6OPPJIqqqq6N27d9bvkRezSKpaDbwKjPM4FGM6hA0bNnDKKadw7bXXctppp7Fq1SrGjh2b9ft4OYvUV0TKIp+XACcDH3gVjzEdyTvvvMNbb73FnDlzePbZZ3Grd+DlLNKewJ9ExE840T2pqgs8jMeYdm379u28+eabnHbaaUyaNImPP/6Yfv36uXpPzxKMqq4Chnl1f2M6ktdee43vfe97bN68mc8++4w+ffq4nlwgT8ZgjDHuaGpq4sYbb+TEE0/E7/ezaNEi+vTpk7P7F9RmR2OMc8FgkOOOO463336bH/zgB9x7771069YtpzFYC8aYdioQCDBp0iSefvpp5syZk/PkApZgjGlXvvrqK84880wWL14MwIwZM5g8ebJn8ViCMaadmD9/PkcccQQLFy5k/fr1XocDWIIxpuDt2LGDyy67jAkTJjBgwACWLVvGtGnTvA4LsARjTMF74oknePDBB/npT3/KW2+9xcCBA70OaSebRTKmAIVCIdauXcugQYOYNm0aQ4YMYcSIEV6HtRtrwRhTYD755BNOOOEEjjvuOKqqqvD5fHmZXMASjDEFQ1V5+OGHGTJkCKtXr+a+++5zZQd0NlkXyZgCEAwGmTp1Kn/96185/vjjefjhh9l33329Disla8EYUwACgQDdunXjjjvuYPHixQWRXMBaMMbkrYaGBm6++WYuvPBCBg0axJw5cxARr8NKiyUYY/LQ6tWrmTp1KqtXr6Zfv34MGjSo4JILWBfJmLzS0tLCvffey8iRI/nqq6+oqKhg+vTpXofVZl5WtNtbRF4RkbWRc5F+7FUsxuSLOXPmcM011zBu3DhWr17N6aef7nVIGfGyi9QMXKuq74pIN2CZiLykqu97GJMxntiyZQu9e/fmwgsvpEePHpx77rkF2SVqzctzkTaq6ruRz78G1gLlXsVjjBdqamqYNm0aw4cPZ/v27RQXFzNlypR2kVwgT8ZgRGQ/wuUz7Vwk02G8/vrrDBkyhEceeYRp06ZRWlrqdUhZ53mCEZGuwNPAVapa0/r7di6SaW+am5u56aabOOGEE/D5fLz++uvcfvvtBAIBr0PLOq9PdgwQTi6PquozXsZiTK74fD7+9a9/ceGFF7JixQqOOeYYr0NyjWeDvBLuZM4F1qrqPV7FYUwuqCpz587ltNNOo7y8nIqKCjp37ux1WK7zsgUzGvgeMEZEVkQ+CntOzrhu3vJKRs9czP4zKhg9czHzlld6HVJKmzZt4swzz+SSSy7h/vvvB+gQyQW8PRfpdaB9DJWbnJi3vJLrn1lNfTAEQGV1Pdc/sxqAicPycwKyoqKCiy66iO3btzN79myuuOIKr0PKKdsqYArGrIXrdiaXqPpgiFkL1+VNgpm3vJJZC9fxRXU9RR/9kw+fupPBgwfz8ssvM2jQIK/DyznPZ5GMceqL6vq0vp5r0RbWhq21KNBYPpw+x1/ATQ8+2yGTC1gLxhSQAWUlVMZJJgPKSpL+XGyrYkBZCdPHHpJxi2fe8kpun7+GbXVBAMpKAqAhvvzn49T9Zwn9p/4SX+eudDnmPGa/8innHH1AyljciNNrlmCM55w+WNPHHrLLGAxAScDP9LGHJLxGW8ZtUsUzb3kl059aSTCkO79W9eUGqhbcTeOG9yk99Di0OYj4w+taoi2sZLEABTe+5ISoaup35YmRI0fq0qVLvQ7DZFHrhw7CSeOOyUfEfbDiPfxAwmvMWrgubqunvKyEN2aMcRRPwCd07VxEdV2QAWUl7Ghspro+3HJRVXaseYWtL90PCL1O/V+6HH7iLkv9e5YGWH7LqYyeuThhLEBacXpNRJap6siU77MEY7yU7KFz8mDNW17JtU+uJBTn33F5gi5V1KczxzuOJxENBdn40FX4Onehz3evpajHHnHfJwJtedQE+CROnF5zmmCsi2Q8lcnAbbS1ES+5QLhFIEC870rk51u3kpwml4b1q+m0x4H4ikvpN+Xn+Et7ID5/wvenSi6J4kw1vpTvbBbJeCrRA+TkwYo3bR3LLxL3oYXwwzxr4bpdvuZk0Z42B9m2eC5f/eV6tv/rSQCKuvZKmlycUHZfFBYdXypk1oIxnko2cJtKslZOScCfNPnE/nzrGaFEmjZ/StX8uwhu/pSuw06nx7HnpYwxHUq4W2ezSMZkSfQBasv0bFlpIG5S8AlJB3ijepQEGHjLC+xoSp6IAOrWvcnm+bPwFXeh79m3UnrgkSl/Jl2x407Rweyrn1hR0MnGEozx3MRh5W16eBKNa3TvHNh5vdato1g1DUFaHA68dtrzIEoPOZZeYy7B36Us7VhTiW21FeKWiERsDMYUnOiGx+hUcWvbI1+fOKycs0aUJ9zwliq57Fj3BlXz70JVKerej75nTHcluZSXlewyLZ9sS0ShsRaMKSjx1qm01qPkm8JNr3ywOeFAbyItjXVsXfQgO95bRKf+B9PSUIu/pFsbI06tdfcn37dEpMNaMKagpJo5AtjR1LxzRiidNS0ADRvWsvGhK9mxZjE9jplC/wtmuZpcAG6fv2aX15nMrOUbSzCmoDhJGMGQcsMzqxg9c3Fa19ZQkKr5s1BV9vivmZQd/z3E734jf1tdcJcp8uljD6EksOu0d6FOWXvaRRKRPwDfBTapasfcbmocm7e8MuGCtNbqgi3UOWy9BKu/pKhbH8QfoN9ZN1PUYw98xbktwB1bciKTmbV84/UYzEPAfcDDHsdh8lTs3iNfkoVz6fKL0NzSQu3KhWxb/Hu6H302ZaPPp1O//bN0h/S0Hl9p68xavvG0i6Sq/wS2ehmDyV/RAd3K6noUEm4JaIsJh3bh6/l3sHXhfRQPOIyug0/J2rVjlQScPWKFOL7ihI3BmLzlZEC3LRo+W8VvL59EzYdL6TnmEvpN+RlF3fpk/T4AvboU8+nM8cyeMnS3cZWoQh1fcSLvE4wdvNZxORnQDfjSL+vs69wV6dqHgy75Nd2PPBMR9x6DaNdn4rBy7ph8xM7SDP5IOYeepQGKi3xc/cSKgiling7PyzVETnVc4GSQ18o1dBw3zVvNn5esj/s9vwgtqgwoK6GuqTnlHiKAxi8/pP6jdygbfT4QruOSi+NZk5WdSLcWTj6xcg3GE06qwaWaHZm3vJJHEyQXAe4+d8jOn9l/RkXSeLQlRM3bz1D92p/xl5bRbfh4/CXdc5JcUnV9CqGIeaaSJhgRuSbZ9zM9ME1E/gKcCPQRkQ3Arao6N5NrmtyKTRhlpQFqG5oJRtbgt95D43SPzayF65KWWYh9b6I6vQDN2zdRVXEPjZ+/R+kho+k19nLXF81FlTuYWm5PK3YTSdWCif5tHAIcCTwXeX0G8M9Mb66q52d6DeOd1gkjXlcl9jey09/YyR6w6BhGNLElSi4aCvLlo9fR0lhL7/FX02XgmJy0WqIxOqnG19Yi5oUkaYJR1dsBRORFYLiqfh15fRvwV9ejM3nN6SxPZXU9+8+oSNgqia3Lkqz1IrCzwHei/UgtjXVIpxLEH6D3uMsp6lVOoKy/s/+hNAV8EGzZ9WvpzAhlUgunUDgdg9kHaIp53QTsl/VoTF5KNG6STlM+2VRCWWmAobe/mHB3dNSxB/Zi4rByRs9cHDe5NKxfRdWCe+lxzDl0G3Y6JQeMcBxfMiUBP2eNKOeVDzbHPbWgrStu29OK3UScJphHgLdF5FnC/1YmYatvO4Rk4ybJxj+cCvhll3GbZN74aCsHXv/33RbcaXOQ6tceoebtZynqNYBO/Q/KKCYIF61q0fCM1Vkjyvm/iUfs9p5snGPUXlbsJuIowajqL0TkeeC4yJe+r6rL3QvLeCnV+EZ03CReE98pgd2OAHGidXJpqlofLmO56WO6Dh1Hz5Muxtcp84Plo/kupMrTyyoZuW+v3WbD2ktRKDels8KoFKhR1V8BG0TEm00bxpFoUab9Z1SktYArdnl+Ml9U16cs6JSIX4RPZo7njRljdhaHaqtQzWZCtVvpe9bN9B57eVaSS2vxij21p6JQbnLUghGRW4GRhGeT/ggEgD8Do90LzbRVJr9dnQ7cRmc62lLQKaTK/jMqGFBWQueAj/rWI6UpNNdupXH9KrocfiIlB4yg/H/mZCWxJNup3Xq8qSNMMWeD0xbMJGACsANAVb/gmylsk2cy+e3q5AGJPa61rWMwSjjxpZtc6v79Jhv/cDlbFv6WUH0NQJuTi0g4qZSXlTB7ylA+mTl+5zR4a62njttTUSg3OU0wTRreU6AAItLFvZBMpjL57ZrqAfGLcMfk8IBn7LnKbmtprKPq779i87P/j6Ie/djzv+/FX9I9o2tOPXqfnV21aMvOabGn9lQUyk1OE8yTIvIAUCYilwCLgDnuhWUykclv13gPTqzzj9474aI5t2goyMaHr2HH6kV0P+Zc+l8wi0DvvTK+7isf7L55NnZTYrR1E29vkNP3dXROZ5HuEpFTgBrC4zC3qOpLrkbWzmVjijORkw7ty6NL1u8ynuD0t2s0hkTnPUcfylyMNai2IOJD/AG6HzmRQO+96Lx39gofJvp/cDp13N6nmLPB6SDvL1X1OuClOF8zaXJrijPRCYUCnDXC+cMwcVg5Vz+xIu73og9lojUwTktaphLc9gVVC+6mxzFTKD3oKLoNHZeFq+7KzfESN3+BFBKnXaR45b5Oy2YgHYkbU5zRpBVvP5ASvzuQTKKHzyfC/jMqqGtq3q0WSzaSi6ry9coX2fjHK2nesgFa3OmGuTle0roSX/QXSHur9eJE0gQjIv8rIquBQ0VkVczHJ0DuRvjaGTemOFONiaR77URjMSFVlMjGRoGyksDOMYhMk0uobjubn/0FW1/4NcUDvsWeF/2W0m8dk+FVd+f2eImtkflGqi7SY8DzwB3AjJivf62qVku3jdzYRZsqgaR77db7ZHwiu43JBENKl+IiVtx6KgCjZy7OaOtA/cfLqP94KT1PuohuR07MeqU5AT6ZOT6r14zH1sh8I+nfoKpuV9VPgV8BW1X1M1X9DAiKyNG5CLA9cmOKM1kCaeu1Jw4r540ZY/hk5viEBbejO6VHz1zMSYf2TToDFU9LsJGGDe8D0GXgSQy4+Hd0P2qyK2Usc7VGxdbIfMPp3+L9QG3M6x2Rr5k2cGOKM1GXpqwkkNa1420xiJ5HlEh0nOHPS9bTkMbUddNXH/Hln65i019vJVRfg4i4Vlohl2tUbI3MN5zuphaNKd6rqi0iknG5TREZR7h15AfmqOrMTK9ZKLI9xZmNrf+JZreKi3yOx1ecvC9cxvLZSBnL7vSdeEPGi+aEcNmHeIPcZSUBbpswMGezOB2hDINTjop+i8gzwKt802r5IXCSqk5s841F/MC/Cc9QbQDeAc5X1fcT/YwV/c6eeNOoyXZQZ4uGgnz15C00rl9N6beOpde4yzNOLlGfzhzPTfNW85e3Piekil+E84/eO26pBZOZbBf9vgz4NXAT4V9SLwOXtj08AI4CPlTVjwFE5HHgTCBhgjHZkailkouVueIPULznt+g66Dt0GfSdrJWx9Iswb3klT7zz+c7xopAqT7zz+W6lFkzuOBqDUdVNqnqeqvZT1T1U9b9UdVOG9y4HPo95vSHytV3YuUjZl2ga1Z/gYe9ZGiDgb3siaGmoDRff/iI8TdvzxO/T9YiTs1ojN6TK7fPXEAztPtN1+/w1WbuPSU+qUwV+qqp3ishviNO9VtUrM7h3vH9d8e7xIPAghLtIGdzPRCSaLg2pUhLw75J8oovpWj+4TjWsX01VxT2Evt5CcflhFA9o+0BnWUkgYXGq8iTV9Zycm2TckaoFszby36XAsjgfmdgA7B3zei/giwyvaRxINF1aXlbCWSPKd2nJBFu0TQ+ohoJse/UhvvrLDYi/iP4XzKLb0LYv/r5g1D6suPXUuEewdtQZmkKQ6lSB+ZH//smFe78DHBypjFcJnAf8lwv3Ma0kqmZ/0qF9eXpZZVYOma9d9RI1bz1F1yFj6TnmYnyd2rYGpDTgY/KIvXjlg807i1QlKsB923Nr4rZwykoCmf7vmDZK1UWaT5KZR1Wd0NYbq2qziFwOLCQ8Tf0HVbXOskti6+z6I6tyo/8tKwnQ1BxKeFSrU6pKqGYTRT32oOuQsRT1Kqdk3yFtvt7sKUMBdhuQfnpZZdy1PbdNGMj0v67cpYB4wCfcNmFgm2MwmUk6TS0iJ0Q+nQz0J1wmE+B84FNVvcHd8HZl09Rtk+wcoYAvnGQcFPVPqrl2K1v+/iuaNn3EgB/cn5UTFKPV5eKNrSQ63Mx2MedGVqapVfUfkYv9XFWPj/nWfBHJ+GRHkxvJNkI6OS4klbp//4stL/wGDTbQ86Qf4OvcNeNrQvK9O5nWcjG54XQdTF8ROSBmzcr+QF/3wjJtkY0D0tKhoSBbX7yf2lUv0mmPA+nz3Z8Q6LN36h90aECSFkyqfT3WkskPThPM1cCrIvJx5PV+wP+4EpFpE7cPSIvLV0So4Wu6jzqbsm9PRfzZG0yNnRlK93hVO7MofzjaKgAgIsXAoZGXH6hqo2tRJdCexmCy/Rs201IJTmlLiJolT1F6+AkEyvrvLGuZiQtG7cPIfXsl/PNI988q0Z+F00PpTWpZ3SogIqXANcC+qnqJiBwsIoeo6oJMA+2I3PgNm4taI8FtG6lacBdNX6wDEXocc25WyipET05M9PCnO65i9Vjyh9N/HX8kfOB9tLzYBuD/XIkoj7X1tMTW2lrxLNn93aw1oqrUrnqJjQ9dSXDLBvqcMZ0ex5ybtetnu9qb1WPJH07HYA5U1Skicj6AqtZLNjeSFIBstjpS/YaN1yWA3deDxN4/k3OiU6ld+QJbF/6W4n2OoM/4qynq3i/r98hm6yLRQkJb7Zt7ThNMk4iU8M3BawcCOR+D8VKyVke6CSZZycxEiSx8xOru97/2yZVc/cSKXVa4ZmsspiXYgC/QmS6HnwSqdB06zpVKc5Dd1oXVY8kfThPMrcALwN4i8ijhM6mnuRVUPspmvz7eb9iAT6hrauaqOMeF1AdDCVsm0WX9sStc4x1dko6WYCPV/3iIhk9X0P/Ce/F16ky3Yae3+XqpuNG6sPUw+SFlgol0hT4gvJp3FOFd0D9W1SqXY8sr2SzU3fo3bI+SADuamjPe9VsfDMVNUOlo+upjqubfRXDLerqNmJC1FkvrI02ir8utddGupUwwqqoiMk9VRwAVOYgpL2WjX59ounX0zMUJyxB8cy8fDcGWrBxqFo9qS7iM5T8fwV/anX7n/oyS/Ydn5dolAX/CDYqmfXPaRVoiIkeq6juuRpPHMu3XJxskTtXNCviE5hZ1LbkA0NJC3bo3KD3oqKyWsQTszOYOzGlN3vcJn0n9KeETBYRw42awq9G1UsgL7ZIt/oL4y+Gj369L0H3KxkmKOz54nc77DsZf0p2Wxh1Ip9KsVpqzxW3tk9OFdk472KcBBwBjgDOA70b+axxKNkic6JiL2VOG8saMMVQnGJvJJLm0NO6gav5dVP1tJjXv/A0AX3GXrCYXwKaGO7hU9WA6Ey74fRDho2LnqmpzLgJrb5INEsfrfp10aF9mLVzH1U+siHuqYiYaPn+PqgV3E/p6Cz2+PTXjRXMiEC+8nqUB6xp1cKlaMH8CRhJOLqcBd2fjpiJyjoisEZEWEUnZzGoP4rVSBDjp0PCm9NhTFE86tC+PLlm/8/D0eMmlre2M2jWv8NVj1yO+IvpPvZOy0ecjPuenMcYeFjd7ylA+nTmee8+NX8by1jOs0FNHl2qQ93BVPQJAROYCb2fpvu8RnvZ+IEvXy3sTh5Wz9LOtPLpk/c6ujfLNPpzYjX2x74nlF6FFtU27o1UVEaFkv6F0GzmBsuMuSLuMZaLxFFvYZhJJlWB2dv4jJS6zclNVXQtkvb+f7175YPNuiaM+GOL2+Wt2eUgTdYZaVPlk5njmLa90vN5FValdXkHdf96i3zm34e/Sk17fuSTlz7UeQE41JW8L20w8qbpIQ0SkJvLxNTA4+rmI1OQiwPZ0LlKigd5tdcGdGxeTTVkrcMD1FY6TS6h2G5ueuo2tL/0OxIcGGwDiVuaPVRLwM3XUPlk9O9t0TKlKZjrvnLciIosI1/Ft7UZV/ZvT67Snc5GSdW2ie5pSdX+cVris+88Stjz/azTYQK9TLqPrsPGICOVxBpXLSgOowvb6oHVvTFZlfIB9Iqp6slvXLlTTxx6SsPURbbkke49T2hxk2+K5+Lv1oc8ZP6FTn312iQGsS2Nyw52tsSauicPKE57RE93TlOw9qTR++SEtwUakKEC/KT9nz/++e5fkkki26twY05onCUZEJonIBsIFrCpEZKEXcXjhtgkDU55MGO89yWhLiOo3H+fLh6+h5l9PAhAo6x+3Rm7rwk43zVvN1U+s2DklHt3CYEnGZINrXaRkVPVZ4Fkv7p0Piou+qe3SszTArWcM3KW7Ev38midXpBxzCVZ/yZYFd9NYuZbSw06g+1GTkr4/dhA50ZR4W+vcGNOadZFyKLrhMXbndEOwJe57Jw4rj7s6Nlbdh2+z8Y9X0FS1nj5n/IS+E6anPJMotrxEsilxq19rssESTA6lW4s3Va2ZQM89Kd7rcAZ8/zd0OfzElPcP+GSXrliyJGL1a002WILJIadV8aKDrvGmq+s/eZetix5AVQn03ps9zrmdoh7OauR27Vy0S7cnURIRbJOiyQ5LMDnkpNp97KBrLG1uYuuiB9n05C00fLqClsYdad+/9a7sRPujpo7ax8ZfTFZ4MsjbUaWqipdo0LVp0ydUzZ9FsGo93UacQdkJ0/AFitO+f+sEZ3uIjNscn+yYDwq54FRUtGxmZXU9/kgZhmhd2ujXY2lzkMoHLgZtoffpV1FywIg23dfvE7oVF9lqXZMVTgtOWYJxQWzt3R4lAUTC3ZNonZcFKzemrMHbXLsVf5cyRHw0bFhLoNcA/KU9Er6/Z8xy/7LSAA3BEPWRGaounfw0NbcQbDXnXVYS4LYJAy3RmLRl9ehY41zr2ruxiaSyup4/L1mf8ho71r7G1oX30ePYKXQ/ajKd9zos6ftLAz6W33Jqwu8nGjCurg/aofDGVTbIm2XxpqKdamncQdWCu6l67pcU9dqLkoNHOfq5YEiTrrxNNh2d7WNbjYllLZgsa+sCtcbKtWx+bhahr6voMfp8ehx7nuNKc8EWTbryNtUObVtUZ9xiLZgsa+sCNW0JIf4i+k/9JWXfnppWGUtIfCoBxJ+OjmWL6oxbLMFkWaqHOVZwy+fULJsPQOe9BzHg4vspLk8+3pLMfgl2Q08cVs4dk4+gZ+numx/tUHjjJusiZVns2pJErQpVpXbF82xbPBfp1JkuA0/C37lr2q2WeGIPdGu9gXLisPKEp0sa4wZLMC6IPrCtF9UBhHZsY8vzv6b+o3fovP9wep9+Ff4UGxTTlWw3tBWaMrlkCcYltz23Zrfkos1BNj58DaEd1fQ8+X/oNnx81g6Xb80Gbk0+8CTBiMgswidDNgEfAd9X1WovYsmW2K5HWWlgl/Uv2hxEigJIUYCeJ15EoM8+dOq7b0b3S3VsrA3cmnzg1SDvS8CgyNnW/wau9yiOrIgurotWhYs9R7px43/44o+Xs+P9VwHocthxGSWXspJA+LCzKUN3nmvd+vAXG7g1+cKrinYvxrxcApztRRzZEm9xnbaEqFnyFNVvPIa/S0/8XXulvE7AJzSrJiw0JYTLacKuYyk2cGvyVT6MwVwEPJHomyJyKXApwD77pC5g7YXW4x3hMpb30Fj5PqWHHU+vU3+YciA3Wjrz6iQnCiQqo2ADtyZfudZFEpFFIvJenI8zY95zI9AMPJroOqr6oKqOVNWRffv2dSvcjLQe72j68kOaNn9K7+9eS58zpjuaJYqWzkw2dvLKB5utGLcpKJ7tphaRC4HLgO+oap2Tn8nX3dTzllfy00ffpPqzNZQeeGT4iw010Ll7WteJlm2IN70dVRLw2ymLxnNOd1N7dWzJOOA6YILT5JLPum5Zy5aHr2DLc3fS0lBLeVkJsy88ntlThuJP4/ztL6rrd666LU/QkrHNiaaQeDUGcx9QDLwk4Qdwiape5lEsbdbQ0MANN9zAvffey6GHHspLL/yd4cOH7/a+ZC2SWLGHr00cVs7+MyriTkXbGhdTKLyaRTrIi/tmU2NjI6NGjWLlypX86Ec/4s4776S0tHS39znZOgDxp5YT7YK2NS6mUNhmxzRFx6yKi4s5//zzqaio4L777oubXKImDivnjRljdluvEiveuEq8jZO2xsUUEkswaaisrGTcuHH84x//AOC6667j9NNPd/zziVoe5WUlCaefo+MxEnmfDfCaQpIP62AKwlNPPcWll15KY2MjGzdubNM1Up0qEI+tcTGFzFowKdTU1DBt2jTOOeccDjroIFasWMF5553XpmtZi8R0NNaCSeHxxx/nkUce4eabb+bmm2/01ZmkAAAGZUlEQVQmENi9aFM6rEViOhJLMHEEg0HWrl3L4MGDufjiizn66KMZMmSI12EZU3Csi9TKunXrOPbYYznxxBPZtm0bPp/PkosxbWQJJkJVeeCBBxg+fDgff/wxv//97+nZs6fXYRlT0KyLRHjR3Nlnn82CBQs45ZRTeOihhxgwYIDXYRlT8KwFQ3jR3B577MHs2bN54YUXLLkYkyUdNsHU1dVx5ZVXsmbNGgDmzJnDj3/8Y3y+DvtHYkzWdcgu0rJly5g6dSrr1q3j4IMPZuDAgV6HZEy71KF+XYdCIe644w5GjRpFbW0tL7/8MldccYXXYRnTbnWoBHP//fdzww03MHnyZFatWsWYMWO8DsmYdq3dd5FUlaqqKvr27csll1zCgAEDmDRpEpJGIShjTNt4VdHu5yKySkRWiMiLIuLKtM3WrVs577zzOPLII6mpqaG4uJjJkydbcjEmR7zqIs1S1cGqOhRYANyS7Ru8/PLLDB48mGeeeYbLLruMLl26ZPsWxpgUPEkwqloT87ILyQ8pTEswGOQnP/kJJ598Ml27dmXJkiXMmDEDvz/zg+WNMenxbJBXRH4hIp8DU8liC8bv9/Puu+/ywx/+kHfffZcRI0Zk69LGmDS5dmyJiCwC+sf51o2q+reY910PdFbVWxNcJ/bgtRGfffZZyns3NTXRqVOnNsVtjEnN6bElnp2LtDMAkX2BClUdlOq9+XoukjEdTb6fi3RwzMsJwAdexGGMcZdX62BmisghQAvwGeETHo0x7YxX5yKd5cV9jTG51aG2ChhjcssSjDHGNZ7PIqVDRDYTHrNJpQ9Q5XI4TuVLLPkSB+RPLBbH7pzGsq+q9k31poJKME6JyFInU2i5kC+x5EsckD+xWBy7y3Ys1kUyxrjGEowxxjXtNcE86HUAMfIllnyJA/InFotjd1mNpV2OwRhj8kN7bcEYY/KAJRhjjGvabYLJVVlOB3HMEpEPIrE8KyJlXsQRieUcEVkjIi0ikvNpUREZJyLrRORDEZmR6/vHxPEHEdkkIu95FUMkjr1F5BURWRv5e/mxR3F0FpG3RWRlJI7bs3ZxVW2XH0D3mM+vBH7nURynAkWRz38J/NLDP5PDgEOAV4GROb63H/gIOADoBKwEDvfoz+F4YDjwnld/F5E49gSGRz7vBvzbiz8TQICukc8DwFvAqGxcu922YNTFspxpxvGiqjZHXi4B9vIijkgsa1V1nUe3Pwr4UFU/VtUm4HHgTC8CUdV/Alu9uHerODaq6ruRz78G1gLlHsShqlobeRmIfGTleWm3CQbcK8uZgYuA570OwiPlwOcxrzfgwcOUr0RkP2AY4daDF/f3i8gKYBPwkqpmJY6CTjAiskhE3ovzcSaAqt6oqnsDjwKXexVH5D03As2RWFzjJBaPxDsrxtZIACLSFXgauKpVyztnVDWk4VM+9gKOEpGUFSadKOiD11T1ZIdvfQyoAOLW/XU7DhG5EPgu8B2NdHTdksafSa5tAPaOeb0X8IVHseQNEQkQTi6PquozXsejqtUi8iowDsh4ELygWzDJ5EtZThEZB1wHTFDVOi9iyBPvAAeLyP4i0gk4D3jO45g8JeETAOcCa1X1Hg/j6Bud3RSREuBksvS8tNuVvCLyNOEZk51lOVW10oM4PgSKgS2RLy1RVU9KhIrIJOA3QF+gGlihqmNzeP/TgdmEZ5T+oKq/yNW9W8XxF+BEwqUJvgJuVdW5HsTxbeA1YDXhf6cAN6jq33Mcx2DgT4T/XnzAk6r6s6xcu70mGGOM99ptF8kY4z1LMMYY11iCMca4xhKMMcY1lmCMMa4p6IV2Jj+ISG/g5cjL/kAI2Bx5fVRk75HpgGya2mSViNwG1KrqXa2+LoT/vbXE/UHTLlkXybhGRA6K7IP6HfAusLeIVMd8/zwRmRP5fA8ReUZElkZqk4zyKm6TPZZgjNsOB+aq6jAg2UrqXwN3avhMnnOBObkIzrjLxmCM2z5S1XccvO9k4JBwTwqAniJSoqr17oVm3GYJxrhtR8znLexatqFzzOeCDQi3O9ZFMjkTGeDdJiIHi4gPmBTz7UXAj6IvRGRoruMz2WcJxuTadcALhKe1N8R8/UfA6Ehx9PeBS7wIzmSXTVMbY1xjLRhjjGsswRhjXGMJxhjjGkswxhjXWIIxxrjGEowxxjWWYIwxrvn/RutV4suBDJQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 288x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#还可以观察预测值与真值的散点图\n",
    "plt.figure(figsize=(4, 3))\n",
    "plt.scatter(y_train, y_train_pred_ridge)\n",
    "plt.plot([-3, 3], [-3, 3], '--k')   #数据已经归一化\n",
    "plt.axis('tight')\n",
    "plt.xlabel('True')\n",
    "plt.ylabel('Predicted')\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**3. 正则化的线性回归（L1正则 --> Lasso）**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 386,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "alpha is: 0.01\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef_lr</th>\n",
       "      <th>coef_ridge</th>\n",
       "      <th>coef_lasso</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>[81468554938586.86]</td>\n",
       "      <td>[0.4484156335610655]</td>\n",
       "      <td>0.315715</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>[81468554938586.69]</td>\n",
       "      <td>[0.2111351597600545]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>[81468554938585.81]</td>\n",
       "      <td>[-0.6595507933213497]</td>\n",
       "      <td>-0.713722</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>workingday</td>\n",
       "      <td>[27391236788852.863]</td>\n",
       "      <td>[0.10166892293776165]</td>\n",
       "      <td>0.029755</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>holiday</td>\n",
       "      <td>[27391236788852.582]</td>\n",
       "      <td>[-0.16688988956593867]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>[26330226186121.223]</td>\n",
       "      <td>[0.12196409869969428]</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>[26330226186121.03]</td>\n",
       "      <td>[-0.056743132071508606]</td>\n",
       "      <td>-0.032015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>temp</td>\n",
       "      <td>[1.6257144485015205]</td>\n",
       "      <td>[1.0063601672600992]</td>\n",
       "      <td>1.497788</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>yr</td>\n",
       "      <td>[1.0310554943013157]</td>\n",
       "      <td>[1.0405499131158658]</td>\n",
       "      <td>1.036710</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>atemp</td>\n",
       "      <td>[0.3700555230824113]</td>\n",
       "      <td>[0.8083373724741407]</td>\n",
       "      <td>0.055660</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>[-0.7869353645955468]</td>\n",
       "      <td>[-0.6813027932711112]</td>\n",
       "      <td>-0.253948</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>hum</td>\n",
       "      <td>[-0.9067522720326273]</td>\n",
       "      <td>[-0.7165574285528961]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>[-1061010602731.6381]</td>\n",
       "      <td>[0.04633234950567558]</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>[-1061010602731.6528]</td>\n",
       "      <td>[0.0210084982163389]</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>[-1061010602731.6815]</td>\n",
       "      <td>[-0.00460898602983515]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>[-1061010602731.7129]</td>\n",
       "      <td>[-0.07503869094688177]</td>\n",
       "      <td>-0.032602</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>[-1061010602731.742]</td>\n",
       "      <td>[-0.052914137373443464]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_4</td>\n",
       "      <td>[-11647555030435.684]</td>\n",
       "      <td>[0.40432968311143325]</td>\n",
       "      <td>0.057178</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_3</td>\n",
       "      <td>[-11647555030436.236]</td>\n",
       "      <td>[-0.015901170727774524]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_2</td>\n",
       "      <td>[-11647555030436.273]</td>\n",
       "      <td>[-0.012402471514923366]</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>season_1</td>\n",
       "      <td>[-11647555030436.523]</td>\n",
       "      <td>[-0.37602604086873226]</td>\n",
       "      <td>-0.654083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>[-20261068421518.26]</td>\n",
       "      <td>[0.2488524783926767]</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>[-20261068421518.312]</td>\n",
       "      <td>[0.32979056524904443]</td>\n",
       "      <td>0.162449</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>[-20261068421518.44]</td>\n",
       "      <td>[0.11437502739429317]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>[-20261068421518.508]</td>\n",
       "      <td>[0.07414051960277476]</td>\n",
       "      <td>0.004907</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>[-20261068421518.527]</td>\n",
       "      <td>[0.10825202290351885]</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>[-20261068421518.547]</td>\n",
       "      <td>[0.12979084687281187]</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>[-20261068421518.613]</td>\n",
       "      <td>[0.06091920668198116]</td>\n",
       "      <td>0.142996</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>[-20261068421518.754]</td>\n",
       "      <td>[-0.10168596947290354]</td>\n",
       "      <td>-0.033518</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>[-20261068421518.816]</td>\n",
       "      <td>[-0.25376846934242697]</td>\n",
       "      <td>-0.048571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>[-20261068421518.84]</td>\n",
       "      <td>[-0.19392861234171235]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>[-20261068421518.9]</td>\n",
       "      <td>[-0.25401852960555443]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>[-20261068421518.92]</td>\n",
       "      <td>[-0.26271908633456986]</td>\n",
       "      <td>-0.006454</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns                coef_lr               coef_ridge  coef_lasso\n",
       "16  weathersit_1    [81468554938586.86]     [0.4484156335610655]    0.315715\n",
       "17  weathersit_2    [81468554938586.69]     [0.2111351597600545]   -0.000000\n",
       "18  weathersit_3    [81468554938585.81]    [-0.6595507933213497]   -0.713722\n",
       "31    workingday   [27391236788852.863]    [0.10166892293776165]    0.029755\n",
       "30       holiday   [27391236788852.582]   [-0.16688988956593867]   -0.000000\n",
       "25     weekday_6   [26330226186121.223]    [0.12196409869969428]    0.000000\n",
       "19     weekday_0    [26330226186121.03]  [-0.056743132071508606]   -0.032015\n",
       "26          temp   [1.6257144485015205]     [1.0063601672600992]    1.497788\n",
       "32            yr   [1.0310554943013157]     [1.0405499131158658]    1.036710\n",
       "27         atemp   [0.3700555230824113]     [0.8083373724741407]    0.055660\n",
       "29     windspeed  [-0.7869353645955468]    [-0.6813027932711112]   -0.253948\n",
       "28           hum  [-0.9067522720326273]    [-0.7165574285528961]   -0.000000\n",
       "24     weekday_5  [-1061010602731.6381]    [0.04633234950567558]    0.000000\n",
       "23     weekday_4  [-1061010602731.6528]     [0.0210084982163389]    0.000000\n",
       "22     weekday_3  [-1061010602731.6815]   [-0.00460898602983515]   -0.000000\n",
       "20     weekday_1  [-1061010602731.7129]   [-0.07503869094688177]   -0.032602\n",
       "21     weekday_2   [-1061010602731.742]  [-0.052914137373443464]   -0.000000\n",
       "3       season_4  [-11647555030435.684]    [0.40432968311143325]    0.057178\n",
       "2       season_3  [-11647555030436.236]  [-0.015901170727774524]   -0.000000\n",
       "1       season_2  [-11647555030436.273]  [-0.012402471514923366]    0.000000\n",
       "0       season_1  [-11647555030436.523]   [-0.37602604086873226]   -0.654083\n",
       "8         mnth_5   [-20261068421518.26]     [0.2488524783926767]    0.000000\n",
       "12        mnth_9  [-20261068421518.312]    [0.32979056524904443]    0.162449\n",
       "7         mnth_4   [-20261068421518.44]    [0.11437502739429317]   -0.000000\n",
       "6         mnth_3  [-20261068421518.508]    [0.07414051960277476]    0.004907\n",
       "11        mnth_8  [-20261068421518.527]    [0.10825202290351885]    0.000000\n",
       "9         mnth_6  [-20261068421518.547]    [0.12979084687281187]    0.000000\n",
       "13       mnth_10  [-20261068421518.613]    [0.06091920668198116]    0.142996\n",
       "10        mnth_7  [-20261068421518.754]   [-0.10168596947290354]   -0.033518\n",
       "4         mnth_1  [-20261068421518.816]   [-0.25376846934242697]   -0.048571\n",
       "5         mnth_2   [-20261068421518.84]   [-0.19392861234171235]   -0.000000\n",
       "14       mnth_11    [-20261068421518.9]   [-0.25401852960555443]   -0.000000\n",
       "15       mnth_12   [-20261068421518.92]   [-0.26271908633456986]   -0.006454"
      ]
     },
     "execution_count": 386,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#### Lasso／L1正则\n",
    "# class sklearn.linear_model.LassoCV(eps=0.001, n_alphas=100, alphas=None, fit_intercept=True, \n",
    "#                                    normalize=False, precompute=’auto’, max_iter=1000, \n",
    "#                                    tol=0.0001, copy_X=True, cv=None, verbose=False, n_jobs=1,\n",
    "#                                    positive=False, random_state=None, selection=’cyclic’)\n",
    "from sklearn.linear_model import LassoCV\n",
    "\n",
    "#设置超参数搜索范围\n",
    "alphas = [ 0.01, 0.1, 1, 10, 100 ]\n",
    "\n",
    "#生成一个LassoCV实例\n",
    "lasso = LassoCV(alphas=alphas, cv = 5) \n",
    "\n",
    "#训练（内含CV）\n",
    "lasso.fit(X_train, y_train.ravel()) \n",
    "\n",
    "#测试\n",
    "y_test_pred_lasso = lasso.predict(X_test)\n",
    "y_train_pred_lasso = lasso.predict(X_train)\n",
    "\n",
    "#输出最佳正则超参数\n",
    "print (\"alpha is:\", lasso.alpha_)\n",
    "\n",
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(columns), \"coef_lr\":list((lr.coef_.T)), \"coef_ridge\":list((ridge.coef_.T)), \"coef_lasso\":list((lasso.coef_.T))})\n",
    "fs.sort_values(by=['coef_lr'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**模型评价**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 387,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of LassoCV  on test is 0.41398555030150774\n",
      "The RMSE of LassoCV on train is 0.43319571511152644\n"
     ]
    }
   ],
   "source": [
    "# 评估，使用 RMSE 评价模型在测试集和训练集上的性能\n",
    "rmse_test = np.sqrt(mean_squared_error(y_test, y_test_pred_lasso))\n",
    "print(\"The RMSE of LassoCV  on test is {}\".format(rmse_test))\n",
    "rmse_train = np.sqrt(mean_squared_error(y_train, y_train_pred_lasso))\n",
    "print(\"The RMSE of LassoCV on train is {}\".format(rmse_train))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 388,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAekAAAFsCAYAAADos0H8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHudJREFUeJzt3XuYXXV97/H3F5I0lpArkxgSaQKCgoRcOmBSDzcjFaESfBCFg5DYAEd79MF6jhX0qNG2Xg4e8UY9UhUiolxSaai2FoxQqBfoBINcQk8iJTUhkiGSYECEhO/5Y6+Jw2QmsyezJ/Ob7PfreebZe+219lrf31qZfOb3W2uvHZmJJEkqz36DXYAkSeqeIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkFbTi4gHI+Kkwa5jMEXEmyLiFxGxLSJm78XtbouIQ3uYtygi/rVB23k0Il7XiHVJe5MhrX1ad/85d/3PPzNflZl39LKeaRGRETFsgEodbJ8G3pWZozLzp11nVm1/ugrVDRHxmYjYv78brbb3SH/XI+2rDGmpAAWE/x8AD/ayzMzMHAWcCLwV+NMBr0pqcoa0ml7n3nZEHBcRbRHxVEQ8HhGfqRa7s3rcUvUm50XEfhHxvyJiXURsioivR8SYTuu9oJq3OSI+1GU7SyJiWUR8IyKeAhZV2/5xRGyJiI0R8cWIGNFpfRkRfxYRayLi1xHxlxFxWPWepyLixs7Ld2ljt7VGxO9FxDZgf+C+iPh5b/srM9cCPwRmdVr/mIj4alX3hoj4q46edkS8PCL+JSK2RsQTEXFDlza9vHo+ISJuqdpyD3BYp+V2GcmIiDsi4sLq+WER8YNqXz8REddFxNge9kVPx1gqjiEtvdjngM9l5mhqIXFj9foJ1ePYaoj2x8Ci6udk4FBgFPBFgIg4Cvgb4DxgMjAGmNJlWwuAZcBY4DpgB/DnwEHAPGA+8Gdd3nMq8IfAXOAvgKuqbbwMOBo4t4d2dVtrZv626h1Drad8WPdv/52IeCVwPLC208tLge3Ay4HZwB8DF1bz/hK4FRgHTAW+0MOqrwSepba//pS+9dQD+ARwMHAktf2xpIdlezrGUnEMaTWDv696p1siYgu18OzJ88DLI+KgzNyWmT/ZzbLnAZ/JzEcycxtwGXBO1dt7M/APmfmvmfkc8GGg643yf5yZf5+ZL2TmbzJzZWb+JDO3Z+ajwJepDS139qnMfCozHwQeAG6ttr8V+CdqAdnXWut1b0Q8DawG7qDajxExCXgD8J7MfDozNwFXAOdU73ue2nD6wZn5bGbucjFY1es+C/hwtY4HqAV/XTJzbWbeVv3R0Q58hl33XYe+HGNpUBnSagZnZubYjh927Z12thg4Ang4Iv4tIv5kN8seDKzrNL0OGAZMqub9omNGZj4DbO7y/l90noiIIyLiOxHxy2oI/OPUetWdPd7p+W+6mR5F93ZXa73mVOt/K/Bq4IDq9T8AhgMbO/0h9GVgYjX/L6j1dO+J2pX03fWQW6p6Ou+Tdd0s162ImBgR11dD7U8B32DXfdehL8dYGlSGtNRJZq7JzHOpBcyngGURcQC79oIBHqMWUB0OoTbk+ziwkdrQLgAR8RJgQtfNdZn+EvAwcHg1FPsBauHWCLurtW5ZcyPwY2qjA1AL1t8CB3X6Y2h0Zr6qes8vM/OizDwY+G/A33Sch+6kvarnZV1q7PB09fj7nV57aafnn6C2P4+p9t3b6GHf7eYYS8UxpKVOIuJtEdGSmS8AW6qXd1ALkReonc/t8C3gzyNiekSMotbzvSEzt1M71/zGiPij6mKuj9J74B4IPAVsq877vrNhDdt9rXvik8DFEfHSzNxI7Zzz/4mI0dVFaodFxIkAEXF2RHT8wfIktTDd0XllmbkD+DawJCJ+vzqnv7DT/HZgA/C2iNi/6o13Pn9+ILCN2oV9U4D39VT4bo6xVBxDWnqxU4EHqyuePwecU51HfQb4a+CH1ZDuXOBrwLXUrvz+D2oXPb0boDpn/G7gemq96l8Dm6j1OHvyP4H/Wi37t8ANu1m2r3qsdU9k5v3Av/C7MLwAGAE8RC2Il1G7AAzgWODuap/eAlySmf/RzWrfRW04/ZfANcDVXeZfVG1vM/Aq4Eed5n2U2nD8VuC71AK/J90e4923WBockdndKJ6kRqp6r1uoDWV3F1CStAt70tIAiYg3VkO3B1C7o9f9wKODW5WkocSQlgbOAmoXbD0GHE5tWNWhK0l1c7hbkqRC2ZOWJKlQe/Wm/gcddFBOmzZtb25SkqSirFy58onMbKln2b0a0tOmTaOtrW1vblKSpKJERN1303O4W5KkQhnSkiQVypCWJKlQe/WctCSpd88//zzr16/n2We9W+lQNnLkSKZOncrw4cP3eB2GtCQVZv369Rx44IFMmzaNiEZ9EZr2psxk8+bNrF+/nunTp+/xehzulqTCPPvss0yYMMGAHsIiggkTJvR7NMSQlqQCGdBDXyOOoSEtSVKhPCctSYVbsmTvr2///fdnxowZbN++nenTp3PttdcyduzYPm/rwgsv5L3vfS9HHXXUi16/5ppraGtr44tf/GKf1wkwatQotm3bVteyJ510Ep/+9KdpbW3do20NJnvSkqRdvOQlL2HVqlU88MADjB8/niuvvHKP1vOVr3xll4BW/QxpSdJuzZs3jw0bNuycvvzyyzn22GM55phj+MhHPgLA008/zemnn87MmTM5+uijueGGG4BaL7bjdtBXX301RxxxBCeeeCI//OEPd65v0aJFLFu2bOf0qFGjANi2bRvz589nzpw5zJgxg+XLl+9S28aNGznhhBOYNWsWRx99NHfddVddbXr00Uc5/vjjmTNnDnPmzOFHP/pRj+vbsWMHixYt4uijj2bGjBlcccUVAKxatYq5c+dyzDHH8KY3vYknn3yy7n1aL4e7JUk92rFjBytWrGDx4sUA3HrrraxZs4Z77rmHzOSMM87gzjvvpL29nYMPPpjvfve7AGzduvVF69m4cSMf+chHWLlyJWPGjOHkk09m9uzZu932yJEjufnmmxk9ejRPPPEEc+fO5YwzznjRBVnf/OY3ef3rX88HP/hBduzYwTPPPFNXuyZOnMhtt93GyJEjWbNmDeeeey5tbW3drm/VqlVs2LCBBx54AIAtW7YAcMEFF/CFL3yBE088kQ9/+MN89KMf5bOf/Wx9O7ZO9qQlSbv4zW9+w6xZs5gwYQK/+tWvOOWUU4BaSN96663Mnj2bOXPm8PDDD7NmzRpmzJjB97//fd7//vdz1113MWbMmBet7+677+akk06ipaWFESNG8Na3vrXXGjKTD3zgAxxzzDG87nWvY8OGDTz++OMvWubYY4/l6quvZsmSJdx///0ceOCBdbXv+eef56KLLmLGjBmcffbZPPTQQz2u79BDD+WRRx7h3e9+N9/73vcYPXo0W7duZcuWLZx44okALFy4kDvvvLOubfeFIS1J2kXHOel169bx3HPP7TwnnZlcdtllrFq1ilWrVrF27VoWL17MEUccwcqVK5kxYwaXXXYZH/vYx3ZZZ08fSRo2bBgvvPDCzvU/99xzAFx33XW0t7ezcuVKVq1axaRJk3b53PEJJ5zAnXfeyZQpUzj//PP5+te/Xlf7rrjiCiZNmsR9991HW1vbzm12t75x48Zx3333cdJJJ3HllVdy4YUX1rcTG8DhbkndqveK4kZfeayyjBkzhs9//vMsWLCAd77znbz+9a/nQx/6EOeddx6jRo1iw4YNDB8+nO3btzN+/Hje9ra3MWrUKK655poXrefVr341l1xyCZs3b2b06NHcdNNNzJw5E6h9jfHKlSt5y1vewvLly3n++eeB2pD5xIkTGT58OLfffjvr1u36DY/r1q1jypQpXHTRRTz99NPce++9XHDBBb22a+vWrUydOpX99tuPpUuXsmPHjh7Xd9pppzFixAjOOussDjvsMBYtWsSYMWMYN24cd911F8cffzzXXnvtzl51IxnSklS4wf5DaPbs2cycOZPrr7+e888/n9WrVzNv3jygdpHXN77xDdauXcv73vc+9ttvP4YPH86XvvSlF61j8uTJLFmyhHnz5jF58mTmzJmzMxgvuugiFixYwHHHHcf8+fM54IADADjvvPN44xvfSGtrK7NmzeKVr3zlLrXdcccdXH755QwfPpxRo0b12JM+/fTTd95De968eXz84x/nrLPO4qabbuLkk0/euc3u1rdhwwbe/va37+ztf+ITnwBg6dKlvOMd7+CZZ57h0EMP5eqrr+7vrt5FZGbDV9qT1tbW7LjKT1LZ7EkPntWrV3PkkUcOdhlqgO6OZUSszMy6PrTtOWlJkgplSEuSVChDWpIKtDdPRWpgNOIYGtKSVJiRI0eyefNmg3oI6/g+6ZEjR/ZrPV7dLUmFmTp1KuvXr6e9vX2wS1E/jBw5kqlTp/ZrHYa0JBVm+PDhTJ8+fbDLUAF6He6OiFdExKpOP09FxHsiYnxE3BYRa6rHcXujYEmSmkWvIZ2Z/56ZszJzFvCHwDPAzcClwIrMPBxYUU1LkqQG6euFY/OBn2fmOmABsLR6fSlwZiMLkySp2fU1pM8BvlU9n5SZGwGqx4ndvSEiLo6Itoho8yIISZLqV3dIR8QI4Azgpr5sIDOvyszWzGxtaWnpa32SJDWtvvSk3wDcm5kdX+b5eERMBqgeNzW6OEmSmllfQvpcfjfUDXALsLB6vhBY3qiiJElSnSEdEb8PnAJ8u9PLnwROiYg11bxPNr48SZKaV103M8nMZ4AJXV7bTO1qb0mSNAC8d7ckSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqHqune3JPVkyZLGLifpd+xJS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUHWFdESMjYhlEfFwRKyOiHkRMT4ibouINdXjuIEuVpKkZlJvT/pzwPcy85XATGA1cCmwIjMPB1ZU05IkqUF6DemIGA2cAHwVIDOfy8wtwAJgabXYUuDMgSpSkqRmVE9P+lCgHbg6In4aEV+JiAOASZm5EaB6nNjdmyPi4ohoi4i29vb2hhUuSdK+rp6QHgbMAb6UmbOBp+nD0HZmXpWZrZnZ2tLSsodlSpLUfOoJ6fXA+sy8u5peRi20H4+IyQDV46aBKVGSpObUa0hn5i+BX0TEK6qX5gMPAbcAC6vXFgLLB6RCSZKa1LA6l3s3cF1EjAAeAd5OLeBvjIjFwH8CZw9MiZIaacmSwa5AUr3qCunMXAW0djNrfmPLkSRJHbzjmCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUaVs9CEfEo8GtgB7A9M1sjYjxwAzANeBR4S2Y+OTBlSpLUfPrSkz45M2dlZms1fSmwIjMPB1ZU05IkqUH6M9y9AFhaPV8KnNn/ciRJUoe6hruBBG6NiAS+nJlXAZMycyNAZm6MiIndvTEiLgYuBjjkkEMaULKkrpYsGewKJA2EekP6NZn5WBXEt0XEw/VuoAr0qwBaW1tzD2qUJKkp1TXcnZmPVY+bgJuB44DHI2IyQPW4aaCKlCSpGfUa0hFxQEQc2PEc+GPgAeAWYGG12EJg+UAVKUlSM6pnuHsScHNEdCz/zcz8XkT8G3BjRCwG/hM4e+DKlCSp+fQa0pn5CDCzm9c3A/MHoihJkuQdxyRJKpYhLUlSoer9CJYk9ctAfJbbz4drX2dPWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKlTdIR0R+0fETyPiO9X09Ii4OyLWRMQNETFi4MqUJKn59KUnfQmwutP0p4ArMvNw4ElgcSMLkySp2dUV0hExFTgd+Eo1HcBrgWXVIkuBMweiQEmSmlW9PenPAn8BvFBNTwC2ZOb2ano9MKW7N0bExRHRFhFt7e3t/SpWkqRm0mtIR8SfAJsyc2Xnl7tZNLt7f2ZelZmtmdna0tKyh2VKktR8htWxzGuAMyLiNGAkMJpaz3psRAyretNTgccGrkxJkppPrz3pzLwsM6dm5jTgHOAHmXkecDvw5mqxhcDyAatSkqQm1J/PSb8feG9ErKV2jvqrjSlJkiRBfcPdO2XmHcAd1fNHgOMaX5IkSQLvOCZJUrEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklSoYYNdgNSMlixp7HLNyv2ofZ09aUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVqteQjoiREXFPRNwXEQ9GxEer16dHxN0RsSYiboiIEQNfriRJzaOenvRvgddm5kxgFnBqRMwFPgVckZmHA08CiweuTEmSmk+vIZ0126rJ4dVPAq8FllWvLwXOHJAKJUlqUnWdk46I/SNiFbAJuA34ObAlM7dXi6wHpgxMiZIkNadh9SyUmTuAWRExFrgZOLK7xbp7b0RcDFwMcMghh+xhmVJzWrJksCvYN9S7H93fKk2fru7OzC3AHcBcYGxEdIT8VOCxHt5zVWa2ZmZrS0tLf2qVJKmp1HN1d0vVgyYiXgK8DlgN3A68uVpsIbB8oIqUJKkZ1TPcPRlYGhH7Uwv1GzPzOxHxEHB9RPwV8FPgqwNYpyRJTafXkM7MnwGzu3n9EeC4gShKkiR5xzFJkoplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIK1WtIR8TLIuL2iFgdEQ9GxCXV6+Mj4raIWFM9jhv4ciVJah719KS3A/8jM48E5gL/PSKOAi4FVmTm4cCKalqSJDVIryGdmRsz897q+a+B1cAUYAGwtFpsKXDmQBUpSVIz6tM56YiYBswG7gYmZeZGqAU5MLGH91wcEW0R0dbe3t6/aiVJaiJ1h3REjAL+DnhPZj5V7/sy86rMbM3M1paWlj2pUZKkplRXSEfEcGoBfV1mfrt6+fGImFzNnwxsGpgSJUlqTvVc3R3AV4HVmfmZTrNuARZWzxcCyxtfniRJzWtYHcu8BjgfuD8iVlWvfQD4JHBjRCwG/hM4e2BKlIaOJUsGuwJJ+5JeQzoz/xWIHmbPb2w5kiSpg3cckySpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqVD1fsCFJTaEvX5Dil6lob7AnLUlSoQxpSZIKZUhLklQoz0lLvfDco6TBYk9akqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIK5fdJS9IeqPd7xv0+cvVHrz3piPhaRGyKiAc6vTY+Im6LiDXV47iBLVOSpOZTz3D3NcCpXV67FFiRmYcDK6ppSZLUQL2GdGbeCfyqy8sLgKXV86XAmQ2uS5KkprenF45NysyNANXjxJ4WjIiLI6ItItra29v3cHOSJDWfAb+6OzOvyszWzGxtaWkZ6M1JkrTP2NOQfjwiJgNUj5saV5IkSYI9D+lbgIXV84XA8saUI0mSOtTzEaxvAT8GXhER6yNiMfBJ4JSIWAOcUk1LkqQG6vVmJpl5bg+z5je4FjWhvtzowZtCSGo23hZUkqRCGdKSJBXKkJYkqVB+wYaalue4JZXOnrQkSYUypCVJKpQhLUlSoTwnrX2O55ol7SvsSUuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSofyctAaEn1WW+sbvVld37ElLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqH8nLT8fKY0gPydUX/Yk5YkqVCGtCRJhTKkJUkqlCEtSVKhhvSFY816wVO9bRmINu9L+1GSSmdPWpKkQhnSkiQVypCWJKlQQ/qc9FDQrOfNJQ2cRv9fMRDXuQyF/88G8/qeevWrJx0Rp0bEv0fE2oi4tFFFSZKkfoR0ROwPXAm8ATgKODcijmpUYZIkNbv+9KSPA9Zm5iOZ+RxwPbCgMWVJkqTIzD17Y8SbgVMz88Jq+nzg1Zn5ri7LXQxcXE2+Avj3PS+3Xw4CnhikbQ8E21M221O+fa1NtqdsndvzB5nZUs+b+nPhWHTz2i6Jn5lXAVf1YzsNERFtmdk62HU0iu0pm+0p377WJttTtj1tT3+Gu9cDL+s0PRV4rB/rkyRJnfQnpP8NODwipkfECOAc4JbGlCVJkvZ4uDszt0fEu4B/BvYHvpaZDzasssYb9CH3BrM9ZbM95dvX2mR7yrZH7dnjC8ckSdLA8ragkiQVypCWJKlQ+2xIR8TlEfFwRPwsIm6OiLE9LDckbm0aEWdHxIMR8UJE9HgZf0Q8GhH3R8SqiGjbmzX2RR/aM1SOz/iIuC0i1lSP43pYbkd1bFZFRHEXWva2vyPi9yLihmr+3RExbe9XWb862rMoIto7HZMLB6POekXE1yJiU0Q80MP8iIjPV+39WUTM2ds19kUd7TkpIrZ2Oj4f3ts19kVEvCwibo+I1dX/b5d0s0zfjlFm7pM/wB8Dw6rnnwI+1c0y+wM/Bw4FRgD3AUcNdu09tOdIajeDuQNo3c1yjwIHDXa9jWjPEDs+/xu4tHp+aXf/3qp52wa71t20odf9DfwZ8H+r5+cANwx23f1szyLgi4Ndax/adAIwB3igh/mnAf9E7T4Wc4G7B7vmfrbnJOA7g11nH9ozGZhTPT8Q+H/d/Jvr0zHaZ3vSmXlrZm6vJn9C7XPcXQ2ZW5tm5urMHKy7tTVcne0ZMseHWl1Lq+dLgTMHsZY9Vc/+7tzOZcD8iOjuxkYlGEr/fuqSmXcCv9rNIguAr2fNT4CxETF571TXd3W0Z0jJzI2ZeW/1/NfAamBKl8X6dIz22ZDu4k+p/eXS1RTgF52m17PrDh1qErg1IlZWt2QdyobS8ZmUmRuh9osKTOxhuZER0RYRP4mI0oK8nv29c5nqj+CtwIS9Ul3f1fvv56xq2HFZRLysm/lDyVD6nanXvIi4LyL+KSJeNdjF1Ks6FTQbuLvLrD4doyH9fdIR8X3gpd3M+mBmLq+W+SCwHbiuu1V089qgfSatnvbU4TWZ+VhETARui4iHq79W97oGtGfIHJ8+rOaQ6vgcCvwgIu7PzJ83psJ+q2d/F3VMelFPrf8AfCszfxsR76A2SvDaAa9s4Ayl41OPe6nd53pbRJwG/D1w+CDX1KuIGAX8HfCezHyq6+xu3tLjMRrSIZ2Zr9vd/IhYCPwJMD+rkwFdFHVr097aU+c6HqseN0XEzdSG/AYlpBvQniFzfCLi8YiYnJkbq6GrTT2so+P4PBIRd1D7S7uUkK5nf3cssz4ihgFjKHe4stf2ZObmTpN/S+36laGsqN+Z/uoccJn5jxHxNxFxUGYW+8UbETGcWkBfl5nf7maRPh2jfXa4OyJOBd4PnJGZz/Sw2D51a9OIOCAiDux4Tu3iuW6vmhwihtLxuQVYWD1fCOwyUhAR4yLi96rnBwGvAR7aaxX2rp793bmdbwZ+0MMfwCXotT1dzgWeQe0c4lB2C3BBdQXxXGBrx2mYoSgiXtpxzUNEHEctszbv/l2Dp6r1q8DqzPxMD4v17RgN9tVwA/UDrKU27r+q+um4IvVg4B87LXcatSvwfk5tGHbQa++hPW+i9hfYb4HHgX/u2h5qV7HeV/08ONTbM8SOzwRgBbCmehxfvd4KfKV6/kfA/dXxuR9YPNh1d9OOXfY38DFqf+wCjARuqn6/7gEOHeya+9meT1S/K/cBtwOvHOyae2nPt4CNwPPV789i4B3AO6r5AVxZtfd+dvNJkBJ+6mjPuzodn58AfzTYNffSnv9Cbej6Z52y57T+HCNvCypJUqH22eFuSZKGOkNakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKh/j8jw0//oNPArQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#在训练集上观察预测残差的分布，看是否符合模型假设：噪声为0均值的高斯噪声\n",
    "f, ax = plt.subplots(figsize=(7, 5)) \n",
    "f.tight_layout() \n",
    "\n",
    "# Reshape 'y_train_pred_lasso' to the same size as 'y_train'\n",
    "y_train_pred_lasso = y_train_pred_lasso.reshape(y_train.shape)\n",
    "\n",
    "ax.hist(y_train - y_train_pred_lasso ,bins=40, label='Residuals Lasso', color='b', alpha=.5); \n",
    "ax.set_title(\"Histogram of Residuals\") \n",
    "ax.legend(loc='best');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 389,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAADQCAYAAADcQn7hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VPW5+PHPk2EICQTCKhJZrHppkUUWFeQqaFUWqywu6OV6oVb90bpjqbjDbblYseq13pctxda1dZcrUEUBtW7IIkukSK+CUCOyBwjZJsnz+2MmYUhmzZwzJ8vzfr3yYmZycs5DYJ75buf5iqpijDFuyPA6AGNM02UJxhjjGkswxhjXWIIxxrjGEowxxjWWYIwxrrEEY4xxjSUYY4xrLMEYY1zTwusAktGpUyft1auX12EY0+ytXbt2r6p2jndco0owvXr1Ys2aNV6HYUyzJyLbEznOukjGGNdYgjHGuMazBCMirURklYhsEJFNIjLbq1iMMe7wcgymDDhPVYtExA98KCJvqupKD2MyxjjIsxaMBhWFnvpDX1acxhgPlJeXu3JeT8dgRMQnIuuB3cA7qvpphGOuF5E1IrJmz5496Q/SmCZu48aNDB48mD//+c+On9vTBKOqlap6GnACcIaI9I1wzHxVHaKqQzp3jjvtboxJUFVVFQ8//DCnn346e/fupWPHjo5fo0HMIqlqIfAeMNrjUIxpFr755hsuuOACbr/9dsaMGcPGjRsZNWqU49fxchaps4jkhh5nAecDX3gVjzHNyerVq/n0009ZsGABr7/+Om71DrycRToeeFpEfAQT3UuqutjDeIxp0g4ePMjHH3/MmDFjmDBhAlu3bqVLly6uXtOzBKOqG4GBXl3fmObkgw8+4Oqrr2bPnj1s376dTp06uZ5coIGMwRhj3FFeXs7dd9/NyJEj8fl8LFu2jE6dOqXt+o3qZkdjTOICgQBnn302q1at4ic/+QmPPPIIOTk5aY3BWjDGNFF+v58JEybw6quvsmDBgrQnF7AEY0yTsmvXLsaNG8eKFSsAmDlzJhMnTvQsHkswxjQRixYtol+/fixdupQdO3Z4HQ5gCcaYRu/IkSNMmzaNSy65hG7durF27VqmTp3qdViAJRhjGr0XX3yR+fPn84tf/IJPP/2UU0891euQatgskjGNUGVlJZs3b6Zv375MnTqVAQMGMHjwYK/DqsNaMMY0Mtu2bWPEiBGcffbZ7N27l4yMjAaZXMASjDGNhqryzDPPMGDAAPLz83n88cdduQPaSdZFMqYRCAQCTJ48mZdffplzzjmHZ555hp49e3odVlzWgjGmEfD7/eTk5DB37lxWrFjRKJILWAvGmAartLSUe++9lylTptC3b18WLFiAiHgdVlIswRjTAOXn5zN58mTy8/Pp0qULffv2bXTJBayLZEyDUlVVxSOPPMKQIUPYtWsXS5YsYcaMGV6HVW9eVrTrLiLvisjm0L5It3gVizENxYIFC5g+fTqjR48mPz+fsWPHeh1SSrzsIlUAt6vqZyKSA6wVkXdU9e8exmSMJ/bt20fHjh2ZMmUK7dq144orrmiUXaLavNwXaaeqfhZ6fBjYDOR5FY8xXjh06BBTp05l0KBBHDx4kMzMTCZNmtQkkgs0kDEYEelFsHym7Ytkmo0PP/yQAQMG8OyzzzJ16lSys7O9DslxnicYEWkDvArcqqqHan/f9kUyTU1FRQX33HMPI0aMICMjgw8//JDZs2fj9/u9Ds1xXu/s6CeYXJ5X1de8jMWYdMnIyOCTTz5hypQprF+/nmHDhnkdkms8G+SVYCfzSWCzqj7sVRzGpIOq8uSTTzJmzBjy8vJYsmQJrVq18jos13nZghkOXA2cJyLrQ1+Ne07OmAh2797NuHHjuO6663jiiScAmkVyAW/3RfoQaBpD5cZEsWTJEq655hoOHjzIo48+yk033eR1SGlltwoY45Knn36aqVOn0r9/f5YvX07fvn29DintPJ9FMqapqaioAGDcuHHMnj2bVatWNcvkApZgjHFMZWUlc+fOZfjw4ZSXl5Obm8t9991HZmam16F5xhKMMQ74+uuvGTlyJHfddRe9evWitLTU65AaBBuDMZ5buK6AeUu38G1hCd1ys5gxqjfjB0a+aySZY9NBVXnuuee44YYbEBGeffZZJk+e3GSW+qfKEozx1MJ1Bdz5Wj4lgUoACgpLuPO1fIA6iSOZY52OMVpSCwQCPPjggzVL/nv16uVaHI2RqKrXMSRsyJAhumbNGq/DMA4a/sAKCgpL6ryel5vFRzPPS+hYnwhVqlFbNAvXFTB70SYOFAcAyM3yM+uSUxNqJbXL8nOkvIJA5dH3SZbfx+SeRdx8xYWs+Oowc17+mD0VmeR1aON5iypdRGStqg6Jd1xCLZjQqtvO4cer6rf1D8+YoG8jJIxor0c7tjL0IVlQWMKtL65n9qJNXNT/eN79Yk/EhFRYEmDGyxuAYMsnPKHkZvspKq0gUKU1x4bTigDfrniSe1e/zrwFl9F+5FQgGzLS16JqTOImGBH5GfCfwD6gKvSyAn1cjMs0E91ysyImgW65WQkfW9uB4gDPrYy9N3OgSpm3dAvAMd2u6lZOJOV7vmbvoocI7PmaNgPH0u6sK+scUxKoZN7SLZZgQuJ2kUTkS2CYqnpeK8G6SE1P7XEVCHZB5k7sF7Grc9uL63GqUy8knrSKt3zMnkXzyMhsTcext5B90ukJXyfbn0Gm30dhcYDcbD+qcLAk0CAGqesr0S5SItPU3wD7Uw/JmLrGD8xj7sR+5OVmIQTHXiIll+pjnRwxTDS5ALQ8/mSye59Ft2seTyq5ABQHqjhQHEAJtpAKS4KPCwpLmPHKBhauK0g++EYikTGYL4EVIrIYKKt+UVUfcy0q06yMH5iX0Kf4wnUFCDiWZM79fmeeX7kj6vmObPmIkn98Qscf3U6Ltl3ofLHzxbcDlcrsRZsaZSsmEYkkmJ2hr7Yux2JMTPOWbnEsubTP9rN4w86I56sqK2b/svkc+XwZLbueQlVpEb6sHIeuXFekcZ+Gtt6nvuImGFW9Nx2BGBNPtFmkZPl9wkX9j484EFz6zWb2LfkNFQd3027YJNoNvwrxpXe5mFfrfdwQ9TcnIr9R1dtF5HUitEpVdaKrkRlTSzJjJrEEKpW/fPrPOq9rZYC9i+aBCMf92wO0OiF9E6XV4zDzlm6J+HdsrLNTsVLzi6E/H3fr4iLyR+BHwG5VbZ63m5qEzRjVu86MU33HZCrDZk8Dhd/RIqcT4vPT5dJ7adHuODIy01uA+87XNgJyzN+tNqdacOkUNcGo6qrQn8tdvP5TBBPYMy5ewzQR1Z/e4WMT8QZqY1FVijYs5cCKP9D2zMvIHX4VLbuc6GzQCSoJVMU9JtLaoIYukYV2JwFzCC6sq6nzp6r/kurFVfVvoS1LjElIpBmneIvqIqksPsi+Nx+j5MtPadXzNNr0v8CpEF2R5fcxY1Rvr8NIWiKjV08BvwIeAsYAP+boil5jPJeX5NhM6faN7Fn0IFWlR2h/3nXkDLkYkYZbuSSvKc8iAdmqulREHlLVr4B7ROQDtwOrJiLXA9cD9OjRI12XNY3Iud/vnFQrJqNVG1rkdKbjpF/RsnMv9wJLUu3xpGgrmhuTRBJMWehmx69EZBpQAHRxN6yjVHU+MB+Ctwqk67omeV6t3Xj3i/h3sZR99yUlX60OjrMc9z26/sfDDapmS5bfx6WD83j3iz2Nfu1LuEQSzG1AG+BmgmMxbYFr3AzKND6R1m7c9uJ61mzfz6/G9zvmuGhJKNL3gLhJK9bsilZVcmjVaxR+8By+7FxyBl2EL6tt2pKLhDVLon06NuYuUDwxb3YUER8wR1VnunJxkb8AI4FOwC7gflV9MtrxdrNjwxWtVosAj0w6raYsQrQbG4E634ukdrdh4boCbntpPZH+G1cc3M3eJQ9T9s/Pye49nA6jbnR1RW5tvgzhN5cPqPm7z3h5Q00ZCAB/hjDpjO6NstXiSD0YVa0UkTOcC6vO+a9y69wmvaK1IhRqFojNW7qlTgKpXkBW/Tie6uPXbN8fc3paKwN89/wdVJUV0fGi22h96nlp7RK1buljzoSjiTDaFPurawuaxIrdaBIp1/AQ8D3gZeBI9euq+oa7odVlLZiGp7pbE28WJ9aCuOq3vRMDbFVlxUjLLESEkq1radEhD39uVwfOnLj22X7W3Xdh3OOSqebX0DhZ0e44goklfFtXBdKeYEzi0jHgGqnLE02s5FG9gCzV2wBKd2xk7+JHaDfscnIGjiXre4NTOl99FcYoWhUumWp+jVWse5FuVNXHVfXqdAZkUpeum+UidXmSFb6ALNFkVZtWBCj84FkOrXqdFh260bLrySnFFE11gaoZo3rz8podfPRV5DJJia64TaaaX2MVqwVzDS7eh2TcE2usw8kEE+uTNt49QuFv1vCYbn1xfVIxlO/dESxjuXsrbU4bTftzryWjpfMby4vAtrkX1TwfPzCPyX/4pE6SSWbFbaR7qxrrit1obNuSJsippnf4+IpPhErVY6ZUo30CV48hxBpjmDGqN/OWbuG2UJFu1boFthNReWgPlUX76XzpvWSffGbSPx+ufbY/ak3eSEOVz183LKWuaKSB38Yyi5SoqIO8IlIBFEf6FqCqmvYCVDbIm5hUBw9rb/NRW6yp5fBp5IXrCpjxyoZjtvzw+4RJp3c/ZvYkWRVF+ynbsZHWfUYCUFVe6kirJVaN3sYw8JpOTtTkzVfVthG+crxILiZxM0b1JsvvO+a1RJve1eM3sarrh08tt/If/S+Um+Wvu7S99ueXwpKNO+udXIr/8TE7/3gj+5b+D5UlhwAc6xJVtyDq+7szdVkXqQlKpemd6MBt9cBx+LGFJQFmvbGpJoZ5S7ccs7AMgtuFxEpe0VSVFbN/+R84kv8OLbueTKcf/RxfVvzPueqK/vGuWZ1EmkO3JZ1iJZiX0xZFM+TGNLIT50xmnCZSIiosCXDna/ms2b7fkepzEFw0t/OZ6VTsL6DtsCvIHX4V4vNHPDY3y09hSaBmkLk4UEVxnFortZfqJ1qE3MQXq+DUf6UzkObEjWlkp86ZG2OgM1Elgcp61WipTbUKkQzE56ft6ePxdzyBVt1jFz48VBqMPdFFeza24q6GWwSjCYu3ZN6rcy5cV0BRaUW9Y3BS4MC3fPfcDIq/XAVAzmmj4yYXgKoklwM3pUVtDZGNwXjAjRWcTpwz0phJuqkqRRvf4cDy+UiGD6pSW8gXT1Na1NYQxVrJOz3WD6rqw86H0zy4sYLTiXNGS0bJbLGaisrig+x767eU/N9KWvXsT8ex02nRtpNr17PZIffF6iLlhL6GAD8F8kJf07CN71PixlSoE+eMloyiTd86rWTrWkq2rqH9udfQZdKvXE0uEafUjeNiDfLOBhCRt4FBqno49HwWNsOUEjemQp0454xRvSMujKs9fetkS6YqUEb5rq9odUIfWp96Lpkn9Kn33c9Zfl9CU+yJ3u1sUpfIGEwPoDzseTnQy5VomhE3pkIdOWeEhXG1zx9tpXCyynd9xd5FD1FxeC95057El9U2pdIKg3q0i3oDYrUsv4/7Lz613tcwyUlkFulZYJWIzBKR+4FPcWgfIxEZLSJbRORLEXGlap5JXLSFcbVnos79fueUrqNVlRxc+Qo7n7mdqrIjdB5/V0KL5uKJlVyE4JS0dYvSK5G9qeeIyJvA2aGXfqyq61K9cKgc5/8AFwDfAKtF5A1V/Xuq527u6rvgLtZMVPg5M1KoDKeVAXa9dB9lO/LJ/pez6DD6RkeSSzzbHrgo/kHGcYlOU2cDh1T1TyLSWUROVNVtKV77DOBLVd0KICIvAOMASzApSGXBXdSZIuGYerKVcaogxiI+P5nH/wtt+v6Q1n1/mJYylj4Pdg/waoeFhiZuFynULboDuDP0kh94zoFr5wHhO5B/E3qt9vWvF5E1IrJmz57421M0d4kuuFu4roDhD6zgxJlLGP7AiuCdz1FmilRJaX1MVWlRsPj2t8EY2o/8MW36nZ+2GrmZLaRmc/l0qE7yBYUlKEeTfDpjaCgSGYOZAFxCqB6vqn5LcPo6VZH+d9X5X6yq81V1iKoO6dw5tb5/c5DIgrtobwCAuRP7OfqJX7ojn2//dBNHNr1H+e6tjp03XOuWsafPiwNVaX2Du7FSu7FKJMGUa7BojAKISGuHrv0N0D3s+QnAtw6du1lauK4g6vhI+BqXeBXvqlLoAlXTygAH3nuKXX+5C/G1oOu/zyPntDEpn7c2AeZM6Bd3jU463+DNodZuohIZg3lJRH4P5IrIdQRLaS5w4NqrgVNE5ESCu0VeCfybA+d1TUPuV1e3SiKNj9RecBfvDeDEqt2ije9w6NNXaDNgFO3Pu5aMlu4sye+Wm1XzbxCrSBak7w3eHGrtJipuC0ZVHwJeAV4FegP3qepjqV5YVSuAG4GlwGbgJVXdlOp53eJ0vzrSGEgqotVx8YnUmZqNtWIX6j8NrapUHNwFQJsBo+hy5Rw6jr7JkeSSm+XH7zu2dRaeOMcPzCO7ZezPy3S9wa1o1VFxWzAi8mtVvQN4J8JrKVHVvwJ/TfU86eBkIW03yjVE+3SuUj1mF8Ron/IZAt8eLKHXzCX1un5F0X72/fW/Kd/9Fd1+8gS+rByyeg6o17mqVe8KCcHff2FJIGJt4GqxWijpfINb0aqjEukiXUBwFincmAivNWlO9qvdqPofr1l+z8L8qDVaMki+zEG44n98wr63fosGSml/7k/IaNWm/icL0y4rWFQqPBlXqtapPlct2u8gUivObVa0KihqF0lEfioi+cD3RWRj2Nc2ID99ITYM8boVyYiXrOrTfYrVLF+4riBmAajY9d6i08oA+958jD2vz6FF284cP+W/yRl0kWPTzwdLAknNyET7HfzmigH2ZvdIrBbMn4E3gblA+DL+w6oa+4aPJsjJPWxitTaidZ/WbN/P4g07a7b2aJ/t5/6LT42593H1p/zwB1YkHWNCMlpQWXqYtkMvI/dfJ0ctY1lf3XKzkmo5Wtek4Ulkb+qhwKawu6lzgD6q+mka4juG19uWODWLFGnL1ertPpK5W9nvE+ZdFv/T+cSZSxzZ9xmC9xEdWvkK2X1G4M/tWlPW0mnxfh9W6tJbTu5N/QQwKOz5kQivNQtO9qszW2TUJJjw1shtSexsGKjUmOM21QnRqeQSOLCTvYsfovzbLSBCu2FXuJJcag/gNvXdD5uyRBKMaFgzR1WrRMRKbdbTPQvzeX7ljmPe9KVhVe+TXYMSfiNi+A6M2f6MuNX0E6WqHMlfxv7l80Ey6HTxDFr3GeHIuWur3TKxbk/jlkgX6TXgPYKtFoCfAeeq6nh3Q6vL6y5SqhauK+C2F9dHbFHkZvlpndmCgsKSOvs6x9rnuX22n9JAVcqb0MdyeP2b7F/6P2T26Eeni26jRdsuSZ/jlC6t+b/dR2IeE74rpGnYnNjZsdo04CyCq22/Ac4Erk8tvOZp1huboiaKwpJATctFOXqjVl5uFpOH9sCfUXdmxu8TVCPvT+SEqkApAK37nEuHC3/GcVfOqVdyyfZn8M70kTw66TSyw3aClND3rFZL05VIPZjdBJfxmxQsXFeQ1ObuyrHdhSE9OzDrjU3HzCJd1P94R/Yfqq0qUEbh+09R+vV6uk55hIyWrcgZOLZe58ry+/iv0D7Wtjak+Ym1q8AvVPVBEfktke9yvtnVyJqY+txoV1BYwokzl9SMO6y//2gd2YXrCpiexIBwosp3bWXvoocI7NtBzuBLUh7EvXSwJZXmLFYLZnPoz8Y76NGAxFrx27qljyPlkbs5tcspVL9ZZ72xqd4L5CJeR6s4tOp1Cv/2LL7stnS54j/JOjH1icJ3v7AaPs1ZrF0FFoX+fDp94TRd0WaHqqeoa0/F1lYSqGTWG5tqZlMc3x6tqoriLR+RffIZjpaxbI4lCsxRsbpIi4ixxa+qXuJKRE1UtJXA4atx4yWPwpJAUuM4iTjyxYe06tkfX1Zbjpv0S6RltqOV5ppjiQJzVKwu0kOhPycCXTlaJvMq4GsXY2qS4q3nqB4AjTWV7aSqsiPsf/sJjvz9PdoOm0T7c64mI7P+tcT8GVBRdewnki2IM7G6SO8DiMgvVfWcsG8tEpG/uR5ZE5TILIqTK28j8YlwZEc+exf/hsrD+2j3r5NpN+yKmD8Tb61N9foVsAVx5liJrMjtLCLfC6v+fyKQUnFcEbkcmAX8ADhDVW0gOcTNMYvcLD8/P2k3//HrO/G160rXyQ+Smff9mD8TvlFZdfJol+VHBAqLAxFbYsZUSyTB3Aa8JyLVFZt7Af8vxet+TrDr9fsUz9PkuLXJvKpSWBLggY1+cgZfQodzrkb9rWL+TO17gix5mGQlUjLzLeAU4JbQV29VXZrKRVV1s6o2vxLrCZgxqnfE7RbqS1U5/Nlidr90H1pVSVFGG9r/8Lq4ySVaUSdjkpHIvkjZwAzgRlXdAPQQkR+5HtnR6zerfZHGD8xj8tAejiSZyqID7H5lFvvf+R1IBhpa+h8u2hYlzXWbDeOsRLpIfwLWAsNCz78BXgYWx/ohEVlGcPaptrtV9X8TDVBV5wPzIXizY6I/15j9anw/hvTskHBtmCx/BuWVSmVY3cvi/1vJvjcfQwOldLhgGm0GRq40V6Ua9WZKW8NiUpVIgjlJVSeJyFUAqloiCSyUUNXzU46uGauecYpVLCpDquvKHLumVysCHFjxJL6cTnS6+Oe07NQj6nVys/0cKqmIuN2JrWExqUokwZSLSBZHN147CShzNSpTIzfbH3WvnyrlmORS9t2X+Dt2J8OfSZdJv6RFTseYZSz9PqGoNHJysTUsxgmJ3Ml2P/AW0F1EngeWA79I5aIiMkFEviHY7VoiIikNGjcmyRT0XriugKLSirjn1KpKCj9+ge+emc6hT14CwJ/btU5yyeDYbVYrKjXqntN2k6JxQswWTKgr9AXBKeWhBEt43KKqe1O5qKq+Dryeyjkao1j7IUHdRWp3v54fd9P5QOF37Fv8G8oKNpP9gxG0PWNC1GMVjrmpMtaZ7SZF44REKtqtVdXBaYonpsZY0S68UHhGqJxlbZFWyvoy5JhB20iKv1zF3kXzQDLoeOFPad1npGNxC7DtgYscO59pWpws+r1SRE5X1dUOxNWs1K6/Gym5ABHHWOIlFwB/++PJPKEPHS+8gRbtkq80F4sN8BonJJJgzgWmicjXBHcUEEBVtb+bgTU2tbc0Off7nesU93ZCybbPKPlqNe1/eD3+jt057vLZDl/BBniNcxJJMGNcj6KRizS24nRy0YpyDrz3FIfXvoG/Y3eqyo7gc2CL1iy/j0sH5/HuF3vsJkXjuFj1YFoRLPh9MsGtYp9U1fhTGs1QpO1NnUwu5bu3sXfRPAJ7d5Az+GJyR0wlw5+Z8nlzs/zMuuRUSybGNbFaME8DAeADgq2YPgTvRTK1uLniVSsC7H55FmgVXS6fTdb3Eh9vzwttvVrdZbNWikm3WAmmj6r2AxCRJ4FV6Qmp8XHjDuiKov34WuciLfx0GjcTf4du+LLbJXUO21rVeC3WQruaqQ3rGsU2Y1Rvsvy++Acm6MjmD9i54KccXr0QgFYn/CDp5NI+29mN6I2pj1gtmAEicij0WICs0PPqWSRnqkI3AbXLYUZb7xJPVdkR9r/zO45sepeWx/cm65Sh9YrH75OaIlHGeClWyUznPpKbgfBymCfOXJL0z5cVbGbPG/OoPLyXdsOvot1ZVyIZyf8T2MCtaUhsE3sX1GdMRqsqEV8Luk7+NZl5P6jXdf99aA9+Nb5fvX7WGDektm2fiSjRqnSBff/k0NpFALTq3pdu1z5R7+QCdv+QaXgswbhg/MC8mOtgVJXD6/7Kzqdu5eDHL1BZWgRQry5ROCsQZRoa6yK5JC9KN6nyyAH2vfkYJV+tptWJg+g49lZ8rdqQASlvBWv3D5mGxlowLok0da0VAXY+M52Sr9fT/vz/R5fLZ9GiTQcAHp50Go9OOi3uebP8GQw/qUOdLpjdP2QaIk9aMCIyD7gYKAe+An6sqoVexOKW8E3qDxwuRlr4kRZ+2o+8Bn+nHrTs3LPm2OEndag5/tYX18c8b4fWmTx/3bA6N1faylzTEHnVRXoHuFNVK0Tk18CdwB0exeKa8QPzOKFyJ+Mu+ykV/S8lq88IWv/g7DrHfb3vaFcqN8sfc//p6nGWRHaJNMZrnnSRVPXtsNXBK4ETvIjDTZWVlcyZM4dhw4aRUVmOtGkf9djwwdlZl5yKPyP6HJSNs5jGpCGMwVwDvBntm41xX6Rt27YxYsQI7rnnHi677DI2btzISf3PjHp8eNIYPzCPeZcPIDer7lJ/G2cxjY1rCUZElonI5xG+xoUdczdQATwf7TyqOl9Vh6jqkM6dU9oSO22eeOVtVq5ZT6cf3c72/tfy/tfFzBjVO2LLxO+TOklj/MA81t9/IY9OOo283CyE4KzU3In9rFtkGpW4NXldu7DIFIL1Zn6oqsWJ/ExDrsm7f/9+PvnkEwLdTuPO1/IpOri/5gbFLL+PuRODK2xnvbGpZoylfbaf+y+2Zf2m8XGyJq/jRGQ0wUHdEYkml4Zs2bJlTJkyhUOHDtFn+nOUBFocc/dz9TasH808z5KJaVa8GoN5HMgB3hGR9SLyO4/iSElpaSnTp0/nggsuoG3btrz//vvsLoucs22VrWmOPGnBqOrJXlzXSWVlZQwdOpQNGzZwww038OCDD5KdnU23t1dEXMFrsz+mOWoIs0iNSvWYVWZmJldddRVLlizh8ccfJzs7G4i8gtdmf0xzZQkmCQUFBYwePZr3338fgDvuuIOxY8cec8z4gXnMndjPZn+MwW52TNgrr7zC9ddfT1lZGTt37ox5rK2yNSbIWjBxHDp0iKlTp3L55Zdz8skns379eq688kqvwzKmUbAEE8cLL7zAs88+y7333stHH33EKaec4nVIxjQa1kWKIBAIsHnzZvr378+1117LmWeeyYABA7wOy5hGx1owtWzZsoWzzjqLkSNHcuDAATIyMiy5GFNPlmBCVJXf//73DBo0iK1bt/KHP/yB9u15fbUgAAAG0UlEQVSj3wFtjInPukgEF81ddtllLF68mAsuuICnnnqKbt26eR2WMY2etWAILpo77rjjePTRR3nrrbcsuRjjkGabYIqLi7n55pvZtGkTAAsWLOCWW24hI6PZ/kqMcVyz7CKtXbuWyZMns2XLFk455RROPdW2WTXGDc3q47qyspK5c+cydOhQioqKWL58OTfddJPXYRnTZDWrBPPEE09w1113MXHiRDZu3Mh5553ndUjGNGlNvoukquzdu5fOnTtz3XXX0a1bNyZMmIBIIpu7GmNS4UkLRkR+KSIbQ8Wm3hYRV6Zt9u/fz5VXXsnpp5/OoUOHyMzMZOLEiZZcjEkTr7pI81S1v6qeBiwG7nP6AsuXL6d///689tprTJs2jdatWzt9CWNMHF7ti3Qo7GlriLlXfFICgQA///nPOf/882nTpg0rV65k5syZ+HypbSxvjEmeZ4O8IjJHRP4JTMbBFozP5+Ozzz7jZz/7GZ999hmDBw926tTGmCS5tm2JiCwDukb41t2q+r9hx90JtFLV+6Oc53rgeoAePXoM3r59e9xrl5eX07Jly3rFbYyJL9FtSzzbF6kmAJGewBJV7Rvv2Ia8L5IxzUmiCcarWaTwqk2XAF94EYcxxl1erYN5QER6A1XAdoI7PBpjmhiv9kW61IvrGmPSq1ndKmCMSS9LMMYY13g+i5QMEdlDcMwmnk7AXpfDSVRDiaWhxAENJxaLo65EY+mpqp3jHdSoEkyiRGRNIlNo6dBQYmkocUDDicXiqMvpWKyLZIxxjSUYY4xrmmqCme91AGEaSiwNJQ5oOLFYHHU5GkuTHIMxxjQMTbUFY4xpACzBGGNc02QTTLrKciYQxzwR+SIUy+sikutFHKFYLheRTSJSJSJpnxYVkdEiskVEvhSRmem+flgcfxSR3SLyuVcxhOLoLiLvisjm0L/LLR7F0UpEVonIhlAcsx07uao2yS+gbdjjm4HfeRTHhUCL0ONfA7/28HfyA6A38B4wJM3X9gFfAd8DWgIbgD4e/R7OAQYBn3v1bxGK43hgUOhxDvAPL34ngABtQo/9wKfAUCfO3WRbMOpiWc4k43hbVStCT1cCJ3gRRyiWzaq6xaPLnwF8qapbVbUceAEY50Ugqvo3YL8X164Vx05V/Sz0+DCwGcjzIA5V1aLQU3/oy5H3S5NNMOBeWc4UXAO86XUQHskD/hn2/Bs8eDM1VCLSCxhIsPXgxfV9IrIe2A28o6qOxNGoE4yILBORzyN8jQNQ1btVtTvwPHCjV3GEjrkbqAjF4ppEYvFIpL1ibI0EICJtgFeBW2u1vNNGVSs1uMvHCcAZIhK3wmQiGvXGa6p6foKH/hlYAkSs++t2HCIyBfgR8EMNdXTdksTvJN2+AbqHPT8B+NajWBoMEfETTC7Pq+prXsejqoUi8h4wGkh5ELxRt2BiaShlOUVkNHAHcImqFnsRQwOxGjhFRE4UkZbAlcAbHsfkKQnuAPgksFlVH/Ywjs7Vs5sikgWcj0Pvlya7kldEXiU4Y1JTllNVCzyI40sgE9gXemmlqnpSIlREJgC/BToDhcB6VR2VxuuPBR4lOKP0R1Wdk65r14rjL8BIgqUJdgH3q+qTHsTxr8AHQD7B/6cAd6nqX9McR3/gaYL/LhnAS6r6n46cu6kmGGOM95psF8kY4z1LMMYY11iCMca4xhKMMcY1lmCMMa6xBGNqiEjH0N3n60XkOxEpCHve0qFrjA07Z1Ho7ur1IvKnJM7RUkTedSIe4y6bpjYRicgsoEhVH6r1uhD8f1MV8QeTu8aHwI2quj7C91qE3SRqGilrwZi4ROTk0P1MvwM+A7qLSGHY968UkQWhx8eJyGsisiZUY2RoEteZJiJ/FpElwGIRaR+ql/JZqJ7OmNBxrURkb+jxaBF5J1RrZ0syLSHjvkZ9L5JJqz7Aj1V1mojE+n/zGPCgqq4M3SG8GEjmxrlhwMDQPTEtgYtVtUhEjiO46jXS3eiDgFOBPcBqERmiqmuSuKZxiSUYk6ivVHV1AsedD/QO9qQAaC8iWapakuB13lLV6taRAA+JyHCgEugZumemtNbPfKyq3wGIyAagF2AJpgGwBGMSdSTscRXHll9oFfZYgDNCRaVSvc6PgWyCLZoKEfkudK3aCaYs7HEl9v+6wbAxGJO00ADvARE5RUQygAlh314G3FD9REROS+FS7YBdoeQyGjguhXMZD1iCMfV1B/AWsJxgrZdqNwDDQ4OyfweuS+EaTwMjRGQ1wfKa21I4l/GATVMbY1xjLRhjjGsswRhjXGMJxhjjGkswxhjXWIIxxrjGEowxxjWWYIwxrvn/AGFe1k0YR0EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 288x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#还可以观察预测值与真值的散点图\n",
    "plt.figure(figsize=(4, 3))\n",
    "plt.scatter(y_train, y_train_pred_lasso)\n",
    "plt.plot([-3, 3], [-3, 3], '--k')   #数据已经归一化\n",
    "plt.axis('tight')\n",
    "plt.xlabel('True Train')\n",
    "plt.ylabel('Predicted Train')\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型系数比较及结果分析"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Linear, Ridge and Lasso Regression 模型系数比较：**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 390,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef_lr</th>\n",
       "      <th>coef_ridge</th>\n",
       "      <th>coef_lasso</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>hum</td>\n",
       "      <td>[-0.9067522720326273]</td>\n",
       "      <td>[-0.7165574285528961]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>[-0.7869353645955468]</td>\n",
       "      <td>[-0.6813027932711112]</td>\n",
       "      <td>-0.253948</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>[81468554938585.81]</td>\n",
       "      <td>[-0.6595507933213497]</td>\n",
       "      <td>-0.713722</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>season_1</td>\n",
       "      <td>[-11647555030436.523]</td>\n",
       "      <td>[-0.37602604086873226]</td>\n",
       "      <td>-0.654083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>[-20261068421518.92]</td>\n",
       "      <td>[-0.26271908633456986]</td>\n",
       "      <td>-0.006454</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>[-20261068421518.9]</td>\n",
       "      <td>[-0.25401852960555443]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>[-20261068421518.816]</td>\n",
       "      <td>[-0.25376846934242697]</td>\n",
       "      <td>-0.048571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>[-20261068421518.84]</td>\n",
       "      <td>[-0.19392861234171235]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>holiday</td>\n",
       "      <td>[27391236788852.582]</td>\n",
       "      <td>[-0.16688988956593867]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>[-20261068421518.754]</td>\n",
       "      <td>[-0.10168596947290354]</td>\n",
       "      <td>-0.033518</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>[-1061010602731.7129]</td>\n",
       "      <td>[-0.07503869094688177]</td>\n",
       "      <td>-0.032602</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>[26330226186121.03]</td>\n",
       "      <td>[-0.056743132071508606]</td>\n",
       "      <td>-0.032015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>[-1061010602731.742]</td>\n",
       "      <td>[-0.052914137373443464]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_3</td>\n",
       "      <td>[-11647555030436.236]</td>\n",
       "      <td>[-0.015901170727774524]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_2</td>\n",
       "      <td>[-11647555030436.273]</td>\n",
       "      <td>[-0.012402471514923366]</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>[-1061010602731.6815]</td>\n",
       "      <td>[-0.00460898602983515]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>[-1061010602731.6528]</td>\n",
       "      <td>[0.0210084982163389]</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>[-1061010602731.6381]</td>\n",
       "      <td>[0.04633234950567558]</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>[-20261068421518.613]</td>\n",
       "      <td>[0.06091920668198116]</td>\n",
       "      <td>0.142996</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>[-20261068421518.508]</td>\n",
       "      <td>[0.07414051960277476]</td>\n",
       "      <td>0.004907</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>workingday</td>\n",
       "      <td>[27391236788852.863]</td>\n",
       "      <td>[0.10166892293776165]</td>\n",
       "      <td>0.029755</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>[-20261068421518.527]</td>\n",
       "      <td>[0.10825202290351885]</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>[-20261068421518.44]</td>\n",
       "      <td>[0.11437502739429317]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>[26330226186121.223]</td>\n",
       "      <td>[0.12196409869969428]</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>[-20261068421518.547]</td>\n",
       "      <td>[0.12979084687281187]</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>[81468554938586.69]</td>\n",
       "      <td>[0.2111351597600545]</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>[-20261068421518.26]</td>\n",
       "      <td>[0.2488524783926767]</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>[-20261068421518.312]</td>\n",
       "      <td>[0.32979056524904443]</td>\n",
       "      <td>0.162449</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_4</td>\n",
       "      <td>[-11647555030435.684]</td>\n",
       "      <td>[0.40432968311143325]</td>\n",
       "      <td>0.057178</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>[81468554938586.86]</td>\n",
       "      <td>[0.4484156335610655]</td>\n",
       "      <td>0.315715</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>atemp</td>\n",
       "      <td>[0.3700555230824113]</td>\n",
       "      <td>[0.8083373724741407]</td>\n",
       "      <td>0.055660</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>temp</td>\n",
       "      <td>[1.6257144485015205]</td>\n",
       "      <td>[1.0063601672600992]</td>\n",
       "      <td>1.497788</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>yr</td>\n",
       "      <td>[1.0310554943013157]</td>\n",
       "      <td>[1.0405499131158658]</td>\n",
       "      <td>1.036710</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns                coef_lr               coef_ridge  coef_lasso\n",
       "28           hum  [-0.9067522720326273]    [-0.7165574285528961]   -0.000000\n",
       "29     windspeed  [-0.7869353645955468]    [-0.6813027932711112]   -0.253948\n",
       "18  weathersit_3    [81468554938585.81]    [-0.6595507933213497]   -0.713722\n",
       "0       season_1  [-11647555030436.523]   [-0.37602604086873226]   -0.654083\n",
       "15       mnth_12   [-20261068421518.92]   [-0.26271908633456986]   -0.006454\n",
       "14       mnth_11    [-20261068421518.9]   [-0.25401852960555443]   -0.000000\n",
       "4         mnth_1  [-20261068421518.816]   [-0.25376846934242697]   -0.048571\n",
       "5         mnth_2   [-20261068421518.84]   [-0.19392861234171235]   -0.000000\n",
       "30       holiday   [27391236788852.582]   [-0.16688988956593867]   -0.000000\n",
       "10        mnth_7  [-20261068421518.754]   [-0.10168596947290354]   -0.033518\n",
       "20     weekday_1  [-1061010602731.7129]   [-0.07503869094688177]   -0.032602\n",
       "19     weekday_0    [26330226186121.03]  [-0.056743132071508606]   -0.032015\n",
       "21     weekday_2   [-1061010602731.742]  [-0.052914137373443464]   -0.000000\n",
       "2       season_3  [-11647555030436.236]  [-0.015901170727774524]   -0.000000\n",
       "1       season_2  [-11647555030436.273]  [-0.012402471514923366]    0.000000\n",
       "22     weekday_3  [-1061010602731.6815]   [-0.00460898602983515]   -0.000000\n",
       "23     weekday_4  [-1061010602731.6528]     [0.0210084982163389]    0.000000\n",
       "24     weekday_5  [-1061010602731.6381]    [0.04633234950567558]    0.000000\n",
       "13       mnth_10  [-20261068421518.613]    [0.06091920668198116]    0.142996\n",
       "6         mnth_3  [-20261068421518.508]    [0.07414051960277476]    0.004907\n",
       "31    workingday   [27391236788852.863]    [0.10166892293776165]    0.029755\n",
       "11        mnth_8  [-20261068421518.527]    [0.10825202290351885]    0.000000\n",
       "7         mnth_4   [-20261068421518.44]    [0.11437502739429317]   -0.000000\n",
       "25     weekday_6   [26330226186121.223]    [0.12196409869969428]    0.000000\n",
       "9         mnth_6  [-20261068421518.547]    [0.12979084687281187]    0.000000\n",
       "17  weathersit_2    [81468554938586.69]     [0.2111351597600545]   -0.000000\n",
       "8         mnth_5   [-20261068421518.26]     [0.2488524783926767]    0.000000\n",
       "12        mnth_9  [-20261068421518.312]    [0.32979056524904443]    0.162449\n",
       "3       season_4  [-11647555030435.684]    [0.40432968311143325]    0.057178\n",
       "16  weathersit_1    [81468554938586.86]     [0.4484156335610655]    0.315715\n",
       "27         atemp   [0.3700555230824113]     [0.8083373724741407]    0.055660\n",
       "26          temp   [1.6257144485015205]     [1.0063601672600992]    1.497788\n",
       "32            yr   [1.0310554943013157]     [1.0405499131158658]    1.036710"
      ]
     },
     "execution_count": 390,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(columns), \"coef_lr\":list((lr.coef_.T)), \"coef_ridge\":list((ridge.coef_.T)), \"coef_lasso\":list((lasso.coef_.T))})\n",
    "fs.sort_values(by=['coef_ridge'],ascending=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上述表格可以看出：\n",
    "\n",
    "1. 线性回归模型中一些相关性很大的特征的系数绝对值很大，并且系数组有很大的方差。\n",
    "   原因是由于特征中存在一些高度相关变量，如体感温度 atemp 和温度 temp 高度相关。\n",
    "   引用 “The Element of Statistical Learning” 中的一段话:\n",
    "        \"When there are many correlated variables in the linear regression model, their coefficients can become \n",
    "         poorly determined and exabit high variance. A widely large positive coefficient on one variable can be\n",
    "         canceled by a similarly large negative coefficient on its correlated cousin.“\n",
    "        \n",
    "        \n",
    "2. Lasso 模型很多变量的系数为0。也就是说它实现了特征的自动选择，去掉了没有信息的特征，也就是把这些特征对应的权重置为0。\n",
    "\n",
    "   Lasso 模型的变量系数反映出 weathersit_1, weathersit_3, workingday, weekday_0, temp, yr, atemp, windspeed, season_4, \n",
    "   season_1, mnth_9, mnth_3, mnth_10, mnth_7, mnth_1, mnth_12 对于最终的输出结果都有影响。\n",
    "   其中，weathersit_1, weathersit_3, temp, yr, windspeed, mnth_9, mnth_10 较其他变量对最终的结果影响更大。\n",
    "    \n",
    "   举例来说，weathersit_1 代表晴天或多云，系数为 0.315715，和结果正相关；\n",
    "   weathersit_3 代表小雪或小雨，系数为 -0.713722，和结果负相关。\n",
    "   同时，结果也符合我们的自然感知，也就是晴天选择骑车的人会增加，雪天会减少，\n",
    "   并且恶劣天气对于减少租车人数的影响比晴天对于增加租车人数的影响更大。\n",
    "    \n",
    "   再来看 weekday_0, weekday_6 和 holiday 的系数，由于这三个变量的强相关性，其中 weekday_6 和 holiday 已被去除。\n",
    "    \n",
    "        \n",
    "3. 从 Ridge 模型的变量系数可以看出，它对于影响小的变量的系数估计值向 0 收缩，但并不会将任何调优参数值严格的变为 0。\n",
    "\n",
    "   其结果反应：hum, windspeed, weathersit_3, season_1, yr, temp, atemp, weathersit_1, season_4, mnth_9 等一些变量\n",
    "   对输出值影响较大，并且和以上在 Lasso 模型中反映出的结果类似。不太一样的是，hum 在 Lasso 模型中被去除，而在 Ridge 模型中显示\n",
    "   其对最终的输出结果影响还是很大的，i.e., 系数约为 -0.716。在 Lasso 模型中被去除的原因很可能是因为 hum 和 weathersit_3 的相\n",
    "   相关性比较大，这也符合它们在 Ridge 模型的变量系数：hum: -0.716, weathersit_3: -0.659, 这两个变量对于结果的影响类似。并且也\n",
    "   符合我们的自然常识，即小雪或小雨的天气，湿度也会很大。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Linear, Ridge and Lasso Regression 模型在测试集上的性能比较**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**三种模型在测试集上的性能比较：**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 391,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0IAAAD9CAYAAACRKsdJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl4lNX1wPHvyWRIJiwJSwAJmxtQEQFBQRF3ZUcUZREXbCsutVWKWPjVCriBRaS2tlaktRZQEFAUUamKooBRQUIRASsgSBAISyCQhExm7u+PycRJMjOZJDN5Zybn8zw8krwz73undY733OVcMcaglFJKKaWUUnVJgtUNUEoppZRSSqnapomQUkoppZRSqs7RREgppZRSSilV52gipJRSSimllKpzNBFSSimllFJK1TmaCCmllFJKKaXqHE2EVMhEpK+IbLe6HUqpukNE/i4ifwhy3YjIWbXZJqWUUvFBEyHll4h8LyJX+/7OGPOpMaajVW1SSsWnknhTICInRGS/iPxLRBoAGGPuNsY8ZnUblVKxzV+/RilNhFTUE5FEq9uglIq4IcaYBkA3oDsw2eL2KKWUinOaCKmQicjlIrLX5+fvReRBEfmviBwTkUUikuxzfbCIZIlIroisE5HzfK5NEpEdIpInIt+IyPU+18aKyFoRmS0iR4CptfUZlVLWMsbsB1biSYgomR163HtdRCaKyI8isk9Efu77XhFpKiLLReS4iHwpIo+LyBqf651E5H0ROSIi20VkRG19LqVU9BGRxiLytojkiMjRkr+39rk+VkR2lvRVdonImJLfnyUiq0v6PodEZJHPey4uiT/HSv55sRWfTYVGEyFVUyOA/sDpwHnAWAAROR/4J3AX0BR4AXhLRJJK3rcD6AukAtOA+SJyms99ewE7gebAExH/FEqpqFDSCRkAfOfnWn/gQeAa4Gyg/DKXvwIngZbA7SV/vO+tD7wPvIInrowG/iYincP/KZRSMSIBeAloB7QFCoDnoDRm/BkYYIxpCFwMZJW87zHgP0BjoDXwl5L3NAFWlLyvKfAMsEJEmtbS51FVpImQqqk/G2P2GWOOAMspGcUF7gReMMZ8boxxGWNeBk4BvQGMMYtL3uc2xiwC/gdc6HPffcaYvxhjio0xBbX4eZRS1lgmInnAD8BBYIqf14wAXjLGfG2MOYnPbLGI2IDhwBRjTL4x5hvgZZ/3Dga+N8a8VBJXvgKWAjdG5uMopaKdMeawMWZpSczIwzPwepnPS9zAuSLiMMb8aIzZUvJ7J57kqZUxptAY4515HgT8zxgzryTOvApsA4bU0kdSVaSJkKqp/T5/zwcalPy9HTChZFlcrojkAm2AVgAicpvPsrlc4Fygmc+9fqiFtiulosewklHXy4FOlI0HXq0oGxt2+/w9HUgsd9337+2AXuVi0hg8s0dKqTpIRFJE5AUR2S0ix4FPgDQRsZUMtowE7gZ+FJEVItKp5K0PAQJ8ISJbfJbptqJsXKLk54zIfxpVHZoIqUj5AXjCGJPm8yfFGPOqiLQDXgTuA5oaY9KAr/EEFS9jQZuVUhYzxqwG/gU87efyj3gGVLza+vw9ByjGs0zFy/e1PwCry8WkBsaYe8LTcqVUDJoAdAR6GWMaAZeW/F4AjDErjTHXAKfhmdl5seT3+40xdxpjWuHZAvC3kjL++/AMuvhqC2RH/JOoatFESAVjF5Fk7x88o62hehG4W0R6iUd9ERkkIg2B+ngSnRwAEbkDz4yQUkoB/Am4RkS6lfv9a8BYETlHRFLwWT5njHEBrwNTS0Z5OwG3+bz3baCDiNwqIvaSPxeIyM8i/FmUUtGjfL+mMZ59Qbkl+3tKY4qItBCRoSV7hU4BJwBXybWbfIoqHMXTp3EB7+CJMzeLSKKIjATOwRN/VBTSREgF8w6eAOH9MzXUNxpj1uPZJ/QcniDxHSWFFErW7s8CPgMOAF2AteFrtlIqlhljcoB/A38o9/t38SRJq/DElFXl3nofngIs+4F5wKt4OjCUrP+/FhiFZ9R2P/AUkIRSqq4o369JAxzAISATeM/ntQl4Zoz2AUfw7B26t+TaBcDnInICeAu43xizyxhzGM9+xAnAYTxL6AYbYw5F+HOpahJjdAWSUkqp+CMiTwEtjTG3V/pipZRSdY7OCCmllIoLJecEnVeyHPdC4BfAG1a3SymlVHSqyp4PpZRSKpo1xLMcrhWeEtyzgDctbZFSSqmopUvjlFJKKaWUUnWOLo1TSimllFJK1TmaCCmllFJKKaXqnJjaI9SsWTPTvn17q5uhVNzasGHDIWNMutXtsJLGGaUiS+OMxhmlIi3UOBNTiVD79u1Zv3691c1QKm6JyG6r22A1jTNKRZbGGY0zSkVaqHFGl8YppZRSSiml6hxNhJRSSimllFJ1jiZCSimllFJKqTrHskRIRJJF5AsR2SQiW0RkmlVtUUrFL401SqlI0zijVGyysljCKeBKY8wJEbEDa0TkXWNMpoVtUkrFH401SqlI0zijVAyyLBEyxhjgRMmP9pI/xqr2KKXik8YapVSkaZxRKjZZukdIRGwikgUcBN43xnxuZXuUUvFJY41SKtI0zigVWePHj+err74K6z0tTYSMMS5jTDegNXChiJxb/jUiMk5E1ovI+pycnNpvpFIxavv27bzzzjtWNyMqVBZrNM4oVT3Z2dmsXr3a6mZEBY0zSkXWNddcw7p168J6z6ioGmeMyQU+Bvr7uTbHGNPTGNMzPb1OH0StVMg+/fRTLrroIu69915OnTpldXOiRqBYo3FGqarbvHkzvXv35uabb6awsNDq5kQNjTNKhc/evXtZuHAhAAMHDuS+++4L6/2trBqXLiJpJX93AFcD26xqj1LxYtGiRVx99dU0b96cVatWkZSUZHWTLKWxRqnw+/DDD7nkkktwu92sWLGC5ORkq5tkKY0zSoWfd7Dlnnvu4ciRIxF5hpUzQqcBH4nIf4Ev8aynfdvC9igV04wx/PGPf2TUqFH06tWLdevWccYZZ1jdrGigsUapMHr55Zfp378/7dq1IzMzk27dulndpGigcUapMFq1ahWXXHIJxhg+/vhjmjRpEpHnWFk17r9Ad6uer1Q82rt3L6NGjeKll16q8yO0XhprlAqvTz75hMsvv5wlS5aQmppqdXOigsYZpcLnlVdeYezYsZx99tm8++67tG3bNmLPsvIcIaVUGJw4cYJ9+/bRoUMHZs+ejYiQkBAV2/+UUnHC6XRy4MABWrduzd///neMMdSrV8/qZiml4tDevXu5+OKLeeONN2jcuHFEn6W9JaVi2I8//shll13Gtddey6lTp7DZbJoEKaXC6tixYwwcOJDLLruMgoIC7Ha7JkFKqbByuVxs2+bZVjdx4kTef//9iCdBoDNCSsWsb775hgEDBnD48GEWLVpU54siKKXCb+/evQwcOJCtW7fy4osv4nA4rG6SUirOFBQUMGbMGFatWsW2bdto2bIldru9Vp6tiZBSMejjjz9m2LBhOBwOVq9eTY8ePaxuklIqzmzatImBAweSl5fHu+++y9VXX211k5RScebQoUMMHTqUzMxM/vSnP9GyZctafb4mQkrFGG91uIyMDN555x3atWtndZOUUnHod7/7HSLCmjVrOO+886xujlIqzuzcuZMBAwawe/duFi9ezPDhw2u9DZoIKRUjjDHk5+dTv359Xn31Vdxud62sn1VK1S1OpxO73c78+fMpLCykdevWVjdJKRWHZs+eTU5ODh9++CF9+vSxpA26q1qpGOB0Ohk3bhzXXHMNhYWFpKamahKklAorYwyPPPII/fv3p6ioiGbNmmkSpJQKO6fTCcDTTz/Nl19+aVkSBJoIKRX18vLyGDJkCHPnzuWqq67SoghKqbArKiri9ttv57HHHqN9+/aIiNVNUkrFoblz59K9e3cOHz5MUlISZ555pqXt0URIqSiWnZ1N3759+eCDD5g7dy6PPfaYdlCUUmGVm5vLgAEDmDdvHo8++ihz586ttYpNSqm6wRjDlClTuPPOO2ndunXUlODXPUJKRbFbbrmFHTt2sGLFCvr162d1c5RScWj06NF88sknvPzyy9x2221WN0cpFWecTid33XUXL730EnfccQcvvPBC1Ay2aCKkVBR74YUXyM/Pp1u3blY3RSkVp5566ilycnK46qqrrG6KUioOTZ48mZdeeokpU6YwZcqUqFrZoomQUlHm5Zdf5rPPPuP555+nQ4cOVjdHKRWH3nvvPT7++GOmT5+upbGVUhE1ceJEunfvzpgxY6xuSgW6R0ipKGGMYdq0aYwdO5YdO3ZQWFhodZOUUnHoxRdfZPDgwaxcuZKTJ09a3RylVBzatm0bd911F8XFxbRo0SIqkyDQREipqFBUVMTPf/5zpk6dytixY1mxYgUOh8PqZiml4ogxhocffri0FP8nn3xCgwYNrG6WUirOrF27lj59+rBs2TK+//57q5sTlCZCSkWB0aNH869//Ytp06bxz3/+M2qqqSil4sddd93FE088wZ133sny5ctp2LCh1U1SSsWZ119/nauuuoqmTZvy2WefcdZZZ1ndpKB0j5BSUWDcuHEMHTqU22+/3eqmKKXi1IABA2jfvj2TJ0+Oqs3KSqn4MHfuXMaNG0fv3r156623aNasmdVNqpQmQkpZJCsriw0bNvCLX/xCS2MrpSJi9+7dfPnll9x4441cf/31XH/99VY3SSkVp7wFEebMmRMzy/t1aZxSFnjvvffo27cvjz32GPn5+VY3RykVhzZs2EDv3r255557OH78uNXNUUrFoVOnTrFw4UIAevTowbx582ImCQJNhJSqdXPnzmXw4MGceeaZrF27lpSUFKubpJSKM++88w6XXXYZ9erV4+OPP6ZRo0ZWN0kpFWeOHj1Kv379GD16NBs3brS6OdWiiZBStegPf/gDd955J1dffTWffvopGRkZVjdJKRVnXnjhBYYMGULHjh3JzMykc+fOVjdJKRVn9uzZQ9++fVm3bh0LFiyge/fuVjepWixLhESkjYh8JCJbRWSLiNxvVVuUqi1paWn88pe/1IpNtUTjjKqL9u/fT//+/Vm9ejWnnXaa1c2pEzTWqLpk06ZNXHTRRfzwww+sXLmSm2++2eomVZuVxRKKgQnGmK9EpCGwQUTeN8Z8Y2GblAq7o0eP8u2339KrVy9++9vfAmjFptqjcUbVCadOnWLHjh2cc845PPLII7hcLhITtR5SLdJYo+qMrVu3YrPZWLNmDV26dLG6OTVi2YyQMeZHY8xXJX/PA7YCuk5IxZXvv/+ePn36MGTIEE6cOIGIaBJUizTOqLrgyJEjXHvttVx22WUcO3YMEdEkqJZprFF1wZ49ewAYNWoU27Zti/kkCKJkj5CItAe6A5/7uTZORNaLyPqcnJzabppS1eat2LRv3z4WLVqkJ7hbTOOMike7du2iT58+ZGZm8pe//IXU1FSrm1TnBYs1SsUiYwwzZsygQ4cOfPnllwBxU+jJ8kRIRBoAS4EHjDEV6nsaY+YYY3oaY3qmp6fXfgOVqoYVK1Zw6aWXkpyczLp167jiiiusblKdpnFGxaP169fTu3dvDhw4wPvvv8+oUaOsblKdFyzW6ICLikUul4tf/epXTJ48meHDh9O1a1ermxRWliZCImLHEzAWGGNet7ItSoXTkiVL6NSpE5mZmZxzzjlWN6dO0zij4tWzzz5LSkoK69at49JLL7W6OXVeZbFGB1xUrMnPz+eGG27g+eefZ9KkScybN4969epZ3aywsmwRsXg2SvwD2GqMecaqdigVLm63m8OHD5Oens4LL7xAUVGRLoezmMYZFY9OnjxJ/fr1mTNnDsePH6dFixZWN6nO01ij4tFLL73E8uXLee655/jVr35ldXMiwsoZoT7ArcCVIpJV8meghe1RqtoKCwsZM2YMffr0IS8vj3r16mkSFB00zqi44Xa7eeihh+jduzfHjx/H4XBoEhQ9NNaouOF2uwG45557yMzMjNskCCycETLGrAG0fJaKeUeOHOG6665jzZo1PPXUU5oARRGNMypeFBYWcvvtt/Paa69x7733Ur9+faubpHxorFHx4osvvuCXv/wlb775JqeffjoXXnih1U2KKK2vqVQN7Ny5k4EDB7Jr1y4WLlzIyJEjrW6SUirOHD58mGHDhrFmzRpmzpzJhAkTtAy/Uirs3n77bUaOHEmLFi1wOp1WN6dWaCKkVA3cf//9HDx4kA8++IC+ffta3RylVBy6++67+fLLL1m0aBEjRoywujlKqTg0Z84c7rnnHs4//3zefvvtOrPsVhMhparB7XaTkJDA3Llzyc3NpWPHjlY3SSkVZss2ZjNz5Xb25RbQKs3BxH4dGda99s/InD17Ng888AB9+vSp9WcrpeLfggULuOuuuxg4cGCdO/fQ8nOElKqOZRuz6TNjFadPWkGfGatYtjG71p79l7/8hSFDhuB0OmnRooUmQUrFoWUbs5n8+maycwswQHZuAZNf31xrsebNN9/klltuweVy0bp1a02ClFIRc9111/Hkk0/y5ptv1qkkCDQRUjHIqg6K2+1mwoQJ/OY3v8Fut1NcXBzR5ymlrDNz5XYKnK4yvytwupi5cnvEn/3cc89x/fXX87///Y+8vLyIP08pVfccP36c8ePHc+LECRo0aMDkyZNJTKx7C8Xq3idWMS9QB2Xa8i0RW8ZSUFDArbfeytKlS/n1r3/N7NmzsdlsYbm3Uir67Mst8Pv77NwC+sxYFZE44y2PPWvWLK677jpeeeUVUlJSwnJvpZTy2rdvH4MGDWLz5s3079+ffv36Wd0ky+iMkIo5gTooR/OdEZsluuWWW3j99dd55plnePbZZzUJUirOtUpz+P29QMTizL333susWbP49a9/zdKlSzUJUkqF3datW7nooov47rvvWLFiRZ1OgkATIRWDAnVQygvnMpaHH36YxYsXM378eC1bq1QdMLFfRxz2sgMeAphyrwtnnLn99tuZPXu2DrYopSLis88+4+KLL6aoqIjVq1fX+SQIdGmcikET+3Vk8uubKyyP88ff7FGolaDWrVvHBx98wCOPPEL37t3ZTfOILYlRSkUX73fbN1ZkB5iNrk6c8V7fvWsH9X7czDPTfsewiy7iQHJbLnnqI40zStUR/mIFEFI/paqVLU877TS6devGSy+9RPv27SP1kWKKJkIq5vjroJw8VUxuQcXDv8rPHnkLLXiTKO/SFt/7LtuYzaRZL7J94ZMkp6Zz5uU3Ub9ho0rfp5SKL8O6Z5T5fveZscpvMuQvzkxcsgmnyzN/lJ1bwMQlm0rv6Y1Dud9/zcGljwHwm/a9eWBRaplZJ40zSsU3f32SiUs2gQGn+6f44S8OhNKf8VqxYgUDBgygffv2fPTRRxH/XLFEl8apmDSsewZrJ13JrhmDWDvpSqYO7VxhGYvDbisdWfGqrBLUG1/t5a7fPcr2BY9Sr8UZNBszk9+/s5MHFmVZVkFKKRUd/C2X8xdnpi3fUpoEeTldhmnLtwCeOHT46085sPD3JCTVp+UtM0lISQX8L70bvyjLkqMClFJlhfvoDn99EqfLlCZBXv76G6FUtnS73YwfP57BgwezYMGCGrU1XumMkIoL/maJ/E0RB1rakp1bwOmTVnD0o39y7IvXSelwMU0HTyDBnhT0uYHup5SKP6HGmaP5FWenfX+/7YNFHPnwReq16kDz4Y9gK0mCAtEZIqWsV5UZmFAFKv4U6LW+S+HKD5qUv2dhYSFXDb2Jde+/TcMeQ3n+h5Y03JitsaMcTYRUxNXW6ezll7H4YxPBZfyHDwPYW55Nw57X0fiKnyMJoW1Wbj9pBRm6ll8pS0VTnAnm9EkrsCWlkNLhopAGW8orcLqY+lbkjgpQSvkXbAamut+/YHsPy0t12EPaH52WYufIkSNcctUAtmZ9QeMrfkGjC69n3/EiHUjxQxMhFVGRGEGpCX9JkOtkLqd+/JaUsy6k/s/6Uv9nfat8X6s/l1J1WTTEGd9ETATKhxq3s5CiAztJbn0OKV2uxnHuVdWuQJlb4CzdE6mxR6naEWj2piqzOuX5K/5kt0mZPULgWYIrQkhFogqdLrZu3cq3W7+m2dCHqP+zS0uveQdSpr61pTSGNE6xM2VI5zobP3SPkIooK09nB0/npNu0/9B+0graT1pB+X6H8/Be9s+bwKG3Z+EuPFGjZ3kDTDjXDyulKhcNcWbikk2l5wuVT4JcJ3M58OrvOfjaH3CdzAVARLCFqRR/gdPFhNc2adxRKoICHd0R6pEe/gzrnsH0G7qQkeZAgIw0BzNv7MrMm7qW+d30G7qQG2DJrS9X/jEKnG5yUtrT6q5/lEmCvHwHUsCzZPeBRVk8vGxztT9HLNMZIRVRkRhBCdWyjdlMXLypzKiKbwelcO8WcpY+DgkJtBjxKAnJDWr8TB2pVar2WRlnwH9xBC/nkWwOLp6C68QRmg15EFv9tNJrbmP8zh5Vh3e2W+OOUpHhb/bGX7GUqvIut/XOKo9flOV3yevMlduDLqMr2PUVOcum03TA/cxc6aBp0yYB9yv6Mz9zD/Mz95Qu9fc+07sE94pO6Xy0LSfuluTqjJCKqEiMoIRq5srtFSqveJ3c+ikHFj5MQkoqGbfNIqnVT4HMYbeR5rCHpQ06UqtU5FkZZyBwcYTCvd+wf96DuE/l02LUk6R0uLjM9VZpjionQY1TKo9NWtFSqfDzN3sz/YYuYUkGvMt7vbPK3gEN3z7DxH4dPcvm/Dix+UMOLplGYmoLkjJ+5immUM0BluzcAiYu3lRmljs7t4D5mXuCti9WaSKkIirUcrO+wlWeMthocFHO9ySddjYtb5lJYlpLGqfYywQ2f+W4q8tlTNwFDqWiSXXiTG0o+PYzEhwNaXnrLJIyOpW55rDbuKJTepWWx2WkOZgyJLTYVFuzYUrVJeWP7gjXjEjIy3vLJTfGGHLXLeTwO7NJbnMuLcf8kcSGTWmV5uCYn7MVQ+V0m4Cz3EHbF4MqTYRE5L5QfqeUP1UdQQllVCRU5UeDjduF8+iPAKT1vYUWI5/A5miId9LIN7D5thugsq5KRppDR2prQOOMqolIjtSGwncG2RiDK/+Y5/eXj6Xlbc9gb3waAAlCafvOb5vKgsw9AatYludN7Mp/1kCJVG3NhsUSjTMqWoWyvNffKpdTP2zm2Kfzqd/5CprfNJWEpBTsCcLEfh1rJQbEw4BLKHuEfg48V+53v/DzuyoTkX8Cg4GDxphza3o/FZ2qUm42nOUpJ/brWLpHyF1UyKG3nuLUj9/S6pd/x+ZoCIk/dV58l7aUL8P7p5HdAHhgUZbf5wiwdtKVFSpXBRIPgSMCNM6oGqlqWetwldv2HaQxbhdHP3yR/G8/47Sxz2Krn4bNZ+9ho2Q7WVOuZdnGbMYvygp4DogvgQrt8/2s/uJONMyGRamIxRmlaiJQGW3fZMa372CMQURIbnsezW+aRvLp5yMipDnsTB36UwW4UPokNW13rAuYCInISGAUcLqIvO5zqRGQG6bn/wtPAPp3mO6nYly4Nz3XT0rkcM5BDi6dRtGBnTS55m5PEhSAvzK8ExdvokFy4DEDbyAof9hiQoAzi+IhcISLxhllhXCV2/a9j7uokEPLZ1Lw3ec0uvAGElIaVXi9d6nKzJXbQ0qCMtIcrJ10ZdDXhHrIa11WG3FGB1xUTYRSiMGbLLlO5pLz1h9pfPkdJJ12Nmedf0lpnChfcGF4jwze3vRjmSpxobCXTF8HWx4XLwMuwWaEvgAOA62Bv/r8Pg/YGI6HG2M+EZH24biXig+hjIqEwttBOfbjLg4unoq74Djpw/9AypkX+H29d2mLvxkpp9sE3AxdPhDoSG2VaZxRtS5cM8/e+7hOHuXg0kcp2r+DxlffRaMeQ/y+3hvHQhnYqUqsqOkhr3VAxOMMOuCiaiCUAY2J/ToyYe57ZL/yMK4TR3Hl55aJE/4GeJZuyCbZXvVyADNv6lqhPfFaNS5gImSM2QXsEpF1QIExxojImUBHKmzXUio8wlWe0ttBOZ65GFxOWtw8g6SWZ/l9bQIwdWhnoGozT2kOOyIwflEWM1durxAUdKS2chpnlBWqOvMcaBmdd9Dm6OqXcebsIf3635Nydi+/9/Cu24fKT5PP0FgRVrURZ3TARdVUZQMaLU/9QM4rEzFFblqMeoKkjE4k2xNYv/tIwNLaBU5XlZfGZaQ5yizDjXeh7BH6BLhURFKB1XhGT0YBt0WyYV4iMg4YB9C2bdvaeKSyULiSh+xDxyHRTpNrf4W78DiJjZoHfG1qir30/pV1UHydKnZXurRGR2pDpnFG1ZqqzDwHWka3fvcRzwFAIjS56k4anj844GALQIPkxNJY4G/AR4Axvdvy+LAuFZ6vgylhY2mcUaq6Zr3yHg+NHUZC/Sa0HDOVxCaeGHA038n8zD1he05dXLUSynxZgjEmHxgOPGeMGQKcF9lm/cQYM8cY09MY0zM9Pb22HqssVJXylOVLbb/x1V6eeOIJ9i/wnN2RUC85aBIElDmtOdQAYBOx9CT7OKRxRtWaqpTbDrSM7sV/LWD/wv/D7TxFQlL9oEkQeDos3pPb/VW5mz2ym98kKFxVNBVgcZwRkXEisl5E1ufk5NTWY1WMW7Yxmxe3uGhw/mDPkR9NqjYQkuawB618axOxpNpmtAhlRihBRC4AbqZkxBQIzwErStVA+ZHavYfzuOMXd3Is6z3qn3M5YgvtUFQDtJ+0onQ5SuMUe9DTmB12W8CpZq0IV20aZ1StqcrMc/nvtDGG418sJffjf5HU+hyMywn2pJCeOz9zD0s37GX6DeeFNFscziqaCrA4zhhj5gBzAHr27KlLf+uI6s7qut1unn76aRYfO4NCl4PGl99R5Wc77DamDu0csOotwKwRXet0PAklEfotMA1YYYz5WkTOAD4Nx8NF5FXgcqCZiOwFphhj/hGOe6voEamlHb6dBPepfHLenEHhrq9odNFI0vreglThoEL4abR1eI8Mlm7IrrBsxfDT2v1A63G1Ily1aZxRtSrUZau+y+iM28WRD17gxMZ3qP+zvjQdOB5JrFel5xY43SFXqAt3FU0VuTijlD90jSzJAAAgAElEQVTVrVC5+Itd3HvXnRzK+pC0y8aS2vvGKj9bgOE9PHEuUJ8lzWGv00kQhJAIGWNWAatEJKnk553AveF4uDFmdDjuo6JXuMrU+uPbGTi88q8Ufp9Fk3730bBb/2rfs8Dp4qNtOUy/oUulyZtWhAsfjTMqWvnu5zn64RxObHyHJhfdSPqVYyksrt49Q53VCVcVTeURyTijAy7Kn+rM6i74ZCvjbr+Z/O+zSLv0Nhr1Gl6tZxvgo22eJZiBClF5C0XVZZUmQiJyIfAPIBVoKyJdgV8aY34d6cap2BfJpR2+nYTGl91Og3OvxHFGjxrdEzwJVmWjxVoRLrw0zqiaitTMs+93vbjHUFIzzqJZz4FBl8+GIpRZnXBV0VQekYwzOuCi/KnqrO6+ffsYN3Iw+Qd303TQeBqce1VYnq99lsBCWRr3ZzyHhC0DMMZsEpErItoqFVZWVh2K5NKOfmk5PLPoX6Reey+Jqc1JTA1eFCFUoY62akW4sNI4o6otkjPPW7du5YvF81jzxBO8mdWRiUta1zgJgtDijHZewk7jjKoV3n5XoI1ggb7/ycnJFEsizW+ciuP07jVuh+9zgvVZ6nJ1ypCKJRhjdpfbb1G1ouTKMpHsIIQinAekTlu+pbQD4t7+Mfve/hOt2p1Ji3rFHHSGtk4/WKED73UdbbWExhlVbeGaeX542WZe/fwHXMZgE+Hi+gd4e9ZvSUpK4t5772Xa8m+CnrTuK6PkAMJXPt+Du9xb9LBUy2icURHn7zB1X/6+/+vXr6dLly40adKEHr/+G/uOFda4HaHGGav7iVYLpXz2DyXTyUZEbCLyAPBthNulwiRYB6E2VKVMbSAPL9vMA4uyOJrvxBhD7ppX+GHZ09Rr3Znp/3yDLx67PmhpSK/GKfbSkrX+2ETqZOnIKKFxJsaVL2VfmyWewzHz/PCyzczP3IPLeLKW41s+Yv7UcbiTU8nMzKR169Bngrwx7vFhXdg5fRB/GtmtTKlsjTOW0TijaiSUOOev3+Xl7/u/cOFC+vTpw5QpUwB4qH8n7LaqFXsqL81hDznOWN1PtFooM0L34JlObgscAD4o+Z2KAVZXHQp1aUegadllG7NZ4HNY2NFVc8lb/yb1z72Kpv3v42/rfmTMpT8jrZKS1wAp9X460NDfunvtnFhK40wMs3pEMRwzz69+/kPp349/8QZHP/oHSW3OpdEND5N11E779qHdRwdUoprGmThg1TKuUONcoP6VAGsnXVl6rz++t41t/3mFox//k3PO78Xvfve7n15cheLqaQ47uQWe/k/jFDtThnSu0v8eVvcTrRYwERKR+4wxzxljDuI5eVnFoGioOlTZ0o5gwaX8GtuUDheRkNyA1ItHISKlX1QTQtDQTYPRR+NMfLD6vJtQiwoE60C5fIKIPb0d9TtfQdP+v0ES7aWfw7fDEYjbmDKf2eokUWmciSdWfp9CjXOV9bseXraZ+et2cWTVXPI2LCelU1+Kr32Q1d/nM6xxY2au3I6z/HraADLSHGWSq5krtzN+URYzV24PuV8TDf1EKwVbGvfzWmuFiphwLE0LJ3/TyoGCy7TlW8jOLaD4eA55m1YCkNzmXNL6jC49I8j7RT1WSefE97XgCZhrJ13JrhmDWDvpSu2QWEfjTBywekRxWPeM0mWvgZafeTtQ2bkFGH7qQD28bDPdH/0P7qIC8v+XCYDj9PNpNngCkmgv8zmmDu2MPSH4kpXynYe6vuwkSmiciRNWfp9CjXMT+3WsECfsCcLEfh1ZtjGb+Zl7KM47xMmvV9HwgmE0GzqRU8ZW+hmqEje9fblA8S2UJcrR1k+sbaEsjVMxLJpmPwKN5ARaS3s030nRgZ0cXDIVt/MUKWf1wlY/rfS6N7BA4BENr0Bf6rpcKUWpcImGEcXyM8/eQRfvdzu/qNhvB2p+5h6KTxwhZ8k0inJ2k3HXHBIbla1A6f0c3vv7Fm7xZbdJhThjdZKoVDyx8vtUpThXfryk5OfJi77AGDuJqS047ed/JbFRs9KXeD9DZf0ZL9/DUGsyKx9N/UQrBEuEzhOR435+L4AxxjSKUJtUmEVL1aFAX1QR/0vbCnZuIOfNGSQk1aflmKfKJEEAM2/qWvq5/C2NETzLbDOC7EvSJSuW0zgTB6LtvBt/3+1AinJ2ewZbCvJofsPvKyRB5T+HN54u25jN1Le2VLo2PxqSRKVxJl7U9vfJd7A01WHHbpMylSP9xbmZK7dXqC7pdBl+9cJ7HFw8hQbnXUtqr+FlkiDfz+AvnpYnUOYw1JomiNHST7RCsERoszGm5kXMVdwINnsSysxKoM6IvyToxH//w+H3nsOe3o7mN04hsWHZgJGR5ihz/+qMaFi9r0EBGmfiQjSNKC7bmM2E1zaV2fMTSOHu/3LwjSdISKxHi5tnkNTyrDLXgxU+CLXjEG1JYh2lcSZO1Ob3qfyASm6BE3uC0DjFTm6+M2Cc85d8nNr/HQeXTAVXMUmt/LfV+xm89wsWx8b0blulfUkqMF0ap0Lib4R14uJNpUtEvLMv3mvjF2XxwKKs0tkYoMxrKmPcbpLbdyP9ukkkJKWUuRYo6FV1REOXrCgVPrU9ouhv8AU8FSFDSYIAig7uJLFBU5rfNLXCgcwCjO7VpsafKZqSRKViXW1+n/wNljrdhpR6iWx85NqA7yuflBTsWO9Z3eJoRItR07E3a+P3ff4Gd/2tdBnTuy2PD+tS5r064FJ9wRKhxbXWChX1AgUE7zr58t0O36Ro4uJNFBvjNwkSKC19bVxOig5+T9JpZ9OwW38adL0WkYr1PJLtoRx/VTkdQYkKGmdUBcHK6c9cuZ3s3IIKgy+TX99MUmJC0OUkAMYYio8dwJ7WkoY9r6NBt/4k2JMrvg5YuiGbnu2ahCUZ0sTHUhpn4khtfZ+qO1jqm5QU5x3i4BuPU69ZO9JvnEJigyYhP78qSZ8OuFRfwETIGPNkbTZE1UykN/3XZJYkWBlIA0wZ0pnfvLyGg288SdGP35Ix7kVsDRr7TYLAU0QhHHt5dATFehpnYkttFBdZtjGbiYs3lcYN72DK+t1HWLohu/T7Wj6qFDhdlSdBrmKO/OdvnNz2Ka1+/hyJqS0QP0mQ7z11qWzs0zgTW6KliFF1B0vLJCU0I33YZJLbdKmwusVXoIPeq5L06YBL9ejSuDhQG5v+Q61iUh2PLfyE46/9nlMHfqDpwPuxNWhc6XvC0UHRERSlQldbxUWmvrWlwuCJ022Y73OwcnW4T+WT8+YMCnd9RaOLRmIrVxQhEF0qq1TtiYYiRg8v28yrn//gd4ltKIOlTqeTd//+KP83dCiDBg3i4WVtg8avqp55psJLE6E4UBub/if268gDi7LCci9fp/Z/x6Yl0zDFRbS9+TEko0vlbyoRjg6KjqAoFZraKi5S2YGl1VGcd4iDSx7FmfM9Tfr/moZd+4X8Xl0qq1TtqY04EyzJeHjZ5oBJi02kzJlF/tpz4sQJRowYwbvvvkvbtm0ZNGhQ6X4eb3IlAo7EBAqcbr9JTjQkg3VJwERIRH4b7I3GmGfC3xxVHbWx6X9Y94wypWLD5eSWj8CWSIuRj9O47VmcKnb7LYHtj3ZQYp/GmdgRy8VF8r58k+LcH2l+4xQanNkzYDGF8vFGR2rjg8aZ2BHpOFNZkvHq5z8EfK83bvi+B35aVdLMVsCR1x/l+2+38MILL9D8gkFlzjKbNaJrSLFCK9rWrmAzQg1L/tkRuAB4q+TnIcAnkWyUqpra2vQ/dWjnSmvbh8pdeIKE5AY0vuLnpF40AltKKscKnMwe2S1gJSjdyxOXNM7EiNqKM41LiqeEg3EVI7ZE0i67jQbnXUuj09pzfttU1u44UuG1t/RuS892TYImOTpSG7M0zsSISMeZQEnG1Le2MHPl9pArThY4XWVWybjyj7Hx37/FnZ/L5Gf+SfMLrq52rIjlQadYFKxYwjQAEfkPcL4xJq/k56loBZaoEulN/74joGkpdpISE6o9M2SMm9zV/+bk1tWcdttsbPXTsKWkAp5AF2ypWmWjsDpSG3s0zsSO2ogz4Zx1PrH5Q45/sZQWo6djS0ml/VkdmNivY+mylvI+2pbD48P8nxnkpSO1sUnjTOyIdJwJlEzkFjhrFHsSHI2o36kvKR0v5uOCNnxcg1ihFW1rVyh7hNoCRT4/FwHtI9IaVS2R3PRffgTUe2ZQPZtQ5Ar1VCBIEHA5nRx6Zzb5Wz+hQbcBJDgall6vLNBVtpdHR2pjnsaZKBfpOONbKa6qbCKlI7nGGI6tW8ixNQtIbncekmADPDHBW3rbn1BGW3WkNuZpnIlykS5ilOqwh3WJf/63n2Fv0hp7szY0vnwsEPjweAgtVmhF29oVSiI0D/hCRN7As3z6euDfEW2VqrJIbfr3NwJqoEpJEIAzP4+c1x/n1N4tpF0+lkYXDkdESq8n2xNYv/tItYOfjtTGPI0zMSCScaa6SRD8tHbfuIo5/N5znPz6A+qfeyVN+/8asdlLX1f+/CFfoYy26khtzNM4EwMiWcTIp9tRY8c3LOfoB3NI6XQJ6df9LqT3hBIrtKJt7ao0ETLGPCEi7wJ9S351hzFmYzgeLiL9gWcBGzDXGDMjHPdV4ROukc6jH/2TUz9up9mQidQ/57KK1/OdZSq1VHVGR0dqY5vGmbotXN/T3NUvc/LrD0i9eDSpl9xcZrDFyxBaUQR/dKQ2tkUyzqjYkFuF/YeNU+wUOt0VB4ONm9yP/8XxL17HcXZvmg68P6T72RMk5FihFW1rj/8TKytKAY4bY54F9orI6TV9sIjYgL8CA4BzgNEick5N76vCq7ojnY1TPKOwpmSktvGVv6Dl6Ol+k6BAfMtUViZQO3WkNqZonKmjwvU9bdRrOM2GTCSt7xi/SZCXwXOAoZT8c/oNwfcGeQ3rnsH0G7pU670qaoQ9zniJSH8R2S4i34nIpHDdV4VPqLHGYbcxZUjn0u+7lyl2cmj50xz/4nUadB9E+rDJJAQ5lNlXg+REjRVRqNIZIRGZAvTEU23lJcAOzAf61PDZFwLfGWN2ljxnIXAd8E0N76tCFEpxgYn9OjJ+UVbAEtb+ZKQ52JdbQP7/Pidv4wqa3/AwtuQG2DJ+VuU2hjpSrCO1sU3jTN1TvghLAuCuwvtFPDM7hQd2cfzLN2na/z5s9dNCGmzJSHOwdtKV1Wq3jtTGrgjGGd9Bl2uAvcCXIvKWMUZjTRTx11fwZ3iPn77nvmcMzVvzHa68w6RdNpZGvYYHHXApL1zVMFV4hbJH6HqgO/AVgDFmn4g0DP6WkGQAvgXb9wK9wnBfFYJgxQWg7NrUi89swrodRyosJTm/barf31/RKZ2//+1vHP5gDvVanom7qABbYr1qtTPU0RtdUxvzNM7EKX8DLkCFIix2m5CUIBQ4PelQ4xQ7p5wu8p3+06NEEY7v+IqcZU+SUC+F4rxD2NNaVtoeuy305Skq7kQqzoAOukQlf/Fn+g1dmPDapqClspduyKZnuyaAp1+xZ88eWqU3RhLr0WL0k6VFWKpCStqj/ZLoEkoiVGSMMSJiAESkfpie7S+NrvBvpYiMA8YBtG3bNkyPVoGKC0xbvqXMmtjs3AKOnCxiTO+2fLQth+zcgtLTlb8/XFD6e2+QubxDU55/aipHMl/HcVYvmg2ZSEK90KaNy6vqjI6O1MY0jTNxyN+AywOLsvwWLHC6DM0bJrN20pWlnZdAI6j2BDia9R8Or3wOe9M2NL9xKomNmoXWqOrXZFCxL1JxBnTQJeoEGvCdfkMXRvdqU2Zfcnm+/aFj2d9xcPFUDrc8kxbDH4FqJEHgCT1awCn6hJIIvSYiLwBpInIn8HNgbhievRdo4/Nza2Bf+RcZY+YAcwB69uyp/wkLk0BLzvx1PAqcLj7allNhSjk7t4ClG7LLrJFv1Wc4RzJfp+H5g2l81Z0hjZoInpmfKzqll0mqdEanTtE4E4f8DbhA4FxkX25Bhc6LP4c/X8bRVXNJbt/ds0Y/KSXkNjndRjsjdVek4gyEMOiiAy61K1g12VAczXdSsHsTOa8/QUI9B2l9b63xOIoWcIo+oVSNe1pErgGO41lX+4gx5v0wPPtL4OySjYrZwCjg5jDcV4UgUBnYQPaVnMHhL6hMeG0T4xdl0SrNgfzsGho7mtGwx5CQ185q0qM0zsSnqv5Hv1WaI2Dy5Cu57Xk06D6IJlfdidhCGc+rWbtUfIhgnIEQBl10wKV21bSa7IktH3H4nWexN2lF85umkdgovcZt0gJO0afSqnEi8pQx5n1jzERjzIPGmPdF5KmaPtgYUwzcB6wEtgKvGWO21PS+KjQT+3XEYQ99erdVSQEEfwqPZHN07avsPZqPvVkbGvUcWqUNhN7p6mUbs0N+j4ovGmfiU1X+o+/duxMozrhPnSQv6z0A6rU4g6bX3lOtJAggQYTTJ62gz4xVGnfqkEjFmRKlgy4iUg/PoMtbYbq3qoZg1WQri01uZyG5n8wjqfXPaDnmj9VKgsr3grSAU3QKpXz2NX5+NyAcDzfGvGOM6WCMOdMY80Q47qlC4y0DG6orOqX7DRynsreyf96D5G1YjivvULXbU+B0MfWtLfSZsUo7KHWTxpk4VKUBl5LxcX9xpvh4DvvnP8SR95/HeeiHCterymUMBh2EqYMiGWd00MUiyzZm++07XNHJf/Jy9OQpruiU7jc2GbcL43aRYE+m5c3TaXHToyQkN6hymxx2G2N6t9VS+zEg4HCaiNwD3AucKSL/9bnUEFgX6YapyPJuRg6Vvz1C+dvXcejtp7E1aErzm6bWeNo4t8BJboFnj1JVD1RVsUnjTPxLSkyodKkbePbuTHhtE6N7tWHphuzS9xQd2MnBJVNxFxXS/KZp2Ju1qeROZdkShIZJiRwrcJIgUqFSlHfPgMaZ+FVbccYY8w7wTrjupyrnryDC+EVZPLAoC1uAlSn5TjdLN2QzvEdG6b5kg2cW6NDyp7E1aEKTa+4hMbVFldvj3fOsy/1jR7B1Ba8A7wLTAd+DwfKMMUci2ioVUaFsRi5vX25BmRLV21Yt5sj7L1CvVQeaD38EW0pq2NupHZQ6QeNMnPIXZ/xVi/PlMqZMB+W7r9Zw6M0ZJNdvSJMRM7Glt8MmQlKiBCyr7atxip0pQzqXxpDTJ63w+zrdMxT3NM7EKX97Cr0xJlh57AKni/mZe8hIczB7ZDfmffRf3p71e4r2fUvjq8eVLu932BNKS/pX5pbebXl8WOgrbVR0CJgIGWOOAcdE5FngiDEmD0BEGopIL2PM57XVSBVewTYj2/yMmIInsDy8bHPpl3zytjYUduxDs0HjEXtSxNqqHZT4pnEmfgXqoHhPaQ9UrMW3SuXvNq8mt3Ermgz/A7aGnvLYLmNwuiFBwF0uVNkThJk3dQ04eBKoSIxuYI5vGmfiV1WKPgV6/6/+/i4HF0/BlXeI9Osnk9LhYgD6nNmEm3q2rXTg2CbC6F5tNAmKUaHsEXoeOOHz88mS36kYFSi5EGDWiK4B1/T/+9Nv6X7nU4xflEVBy66kD5sUUhIkwJ9GdiuzVvaWcmtnG6fY/b5XOyh1hsaZOBMozmTnFpBfVBzwfcYYdm7bzPhFWZxqcyEtb3umNAnycrpMhZmlNIc9aBIE/vcs6QbmOkXjjIUC7eWpyf1CL8vkn3E5ObDoYdwFeTQf+URpEgSwZV8eAMn2wF3ljDQHO6YP1CQohoVSckeM+WmKwBjjFpHqlepRUSFY6ewHFmX5/b3rZC4Hlz7GDwd2kDFuDompzav0vMoOO/W3jEY7KHWKxpk4EyzOBDoo1bicHH73z5zc+gmnjf0z9dLbBTyLzHfi2mG3MXVo50qX0fou79XzyuokjTMWCXS4KVR/H/DMldtrfK6P2Ow0vfZeElNbYG/ausy13AInExdvwll+6tmHrlqJfaHMCO0Ukd+IiL3kz/3Azkg3TEVOoEpOgb7qzsN72T//QZw535N+3e+qlAQB5BcVVzry461ipxVW6iyNM3GmqiX63YUnOPDaFE5u+Yi0Pjdjbxb6gZNVOSRxWPcM1k66kl0zBrF20pUaY+oWjTMWqenhpv7UJAnJy3qvtBy/44weFZIgr2BJEOiqlXgQykjI3cCfgYfx9JU/pORkZBW7fCs5+Vtr71W4dws5Sx+HhARajH6SpFZVn6E5mu8MaeSnslkjFdc0zljEW0Ey3DMk5Wdf/FVs8yo+dpCDi6fiPLqPpoMn0KDzFVV+XnZuAX1mrNIZHhWMxhmLBJodzs4tYNnG7Gp9Z6t6MDx4lt4eW7OAY+sW4jjzAhp07Velcw996aqV+FBpImSMOYjnYDAVB/wtQQs24HEqexsJKY08ZWvTWob0DH8FF7QCnApG44w1IrFcxffevglWsA7Lya2rKT5xmBYjHiW53XnVfqaW3VfBaJyxhncvT6CuRnW/s+WP9KiMcRVz+L3nOPn1B9Tvcg1N+/2q2kmQAMN76OBtPAh2jtBDxpg/ishf8PPvrzHmNxFtmYrISG2winFexhhceTkkNmpOowtvoGH3gSTUC236124TnC7/4U7X0qryNM5YK9hylZrEGn8Jlr+OkLuogIR6Dhr1upH651xGYqPQlt3aEyTgkhUddFHlaZyxVmV7ear7nfW+PtDeZl/G7eLg0kcp3PUVqX1uJrXP6GonQeD5l+ijbTnVfr+KHsFmhLaW/HN9bTRElRWpkdrKkhHjdnHkgxfI/2Y1p93xFxJTmyMhJkEA9eslUj8pUUvUqlBpnLFQoHhQ00GLQKWzfZOhvKz3OLb2FVqM+SP2tJYhJ0EADZITSannP86ADrqoCjTOWCiU72N1l8gN657BzJXbK10iJwk2ktt2IaXjJTTsem2VnhGIxpn4EOwcoeUl/3y59pqjvCI1UhtoiUqaw46rqJAdSx6nYMeXNOo1HFujZn7uENyxAidTh3bWCnAqJBpnrBWpc3UCdRAM0KpREt8sn8OxzMUkn9GjWocx5+Y72fjItfSZsUoHXVSlNM5YK9S9PJFYIuc89AOuwhMkt/4Zqb1vqtJ9K6NxJj4EWxq3nCCHgBtjhkakRQqI3Eitv4DhsNu4/+J0/jr5Tk7t2shdk5/k8+QeAUvcBuMtlQ1aolZVTuOMta7olM78zD1+f18TgTo+pzWw0f7rf/JZ5mLSug+g0dV3ByyPDYELuXg7IIHimQ66KF8aZ6wV6l6emi6R884MiXjK63uKPT2GrX5jTvv5c0FjTXl2m4AJXDVO40z8CLY07umSf94AtATml/w8Gvg+gm1SRG6k1l+SckWndH4/bRr7//s17UdO4UPpijOEJMhhtwXsgGgFOBUijTMWCrTGvaZr3/11fOwJwq4PF5D58as0uex2GvS6MegafYfdxvAeGSzdkB00zoAOuqhKaZyxUPnvaVqKPeBAa3UHe73P8Madk9vXcmj50ySmtiD9xil+k6Dy+xa9P2eUxBHfNqc67Ih4ZqM1zsSXYEvjVgOIyGPGmEt9Li0XkU8i3rI6LlIjtVA2SVn65W4efmsr9t630LLjFbjT2+MOUOzAlzdQaAdE1YTGGWsF6nR4S1FX97tdvuOT6rBzsqiYeucPI73ZmaSceUHQ92f4PLNnuyZB44wOuqjKaJyxXvnvaSSWtXq3FBxf/yZHP5xLUkYn0of/AZujUYXXegdaPtqWEzTOaWyJf6GcI5QuImcYY3YCiMjpQM174yqoQCOy8zP38OrnP+AypkxnoToWLFjAuIlTaDLiCWzJDaiX3j6k93lHZLUDosJI44wFAs08Cz+d+1HdQi3e+LB+/XquufleGg6aSEJSStAkKCPNwdpJV/q9j1JhoHEmSkRiWeu+3AKMMZzK3oajQ2+aDX6QBHtShdelOexMHdpZ44oCQkuExgMfi4j39OX2wF0Ra5ECgk8Pe8/oqW4HxRjD9OnT+f3vf09Sm3NDeo9NBLcxOvOjIkXjjAUCrd0vPyccaO1+ZSX+3377bUaOHMmpxAak5OeSkJQSsC265l7VAo0zUaIqy1pDOUqksLCQZgn55LhTaDbot5CQULocTvsvKphQDlR9T0TOBjqV/GqbMeZUZJulQq2yUtUOSnFxMffeey8vvvgiN998Mzt/dis/ngi+gdFhtzH9hi4aPFTEaJyxRoVNxgTeUV5+cKayEv/PP/889913H927d0f6TSLHVXHJi3ZQVG3SOGMtf/2S8jPA/t5T2VEiR48eZdiwYRzIPkDKiJkUJtpL36/9F1WZShMhEUkBfgu0M8bcKSJni0hHY8zbkW9e3VWVE5Or0kFZs2A2L774Iv/3f//HY489xlubfvS7qblBcqJuClS1RuOMdbxLzwKt2fcqv3Y/WIn/3Z8s4YEHHmDw4MG8+uqrfPC/Y36XwWgHRdUmjTPWCdQvWb/7SNB9OpUdJbJnzx769+/Pjh07ePnll0nueL7uXVZVEsrSuJeADcBFJT/vBRYDGjgiqPxIbTBV6aAsffBBunbtyq233lrhORo4lIU0zljAd4Q2WIkUf8vWgpX4v/7m6zlw4ACPPvooiYmJDOveANA4oyyncSZCgi1fW7YxmwmvbSpd1u9V4HSxIHNPaezxN9sTLM5kZWUxcOBA8vPzWblyJZdffnmZ9yoVilASoTONMSNFZDSAMaZAgtU8VWHjHaktP5LiK5QOSlHO9+RtWI659l4yf3SVJkHln6OUhTTO1LJgccWXTcTvzE355buugjzyvnqbcwaMpW3btjz55JNlXq9xRkUBjTMREGwVCnhKWpdPgrwq248Y8Eyy1GTuv/9+bDYba9eupXPnzuH5MKrOSQjhNUUi4qDk31cROROo0ZpaEblJRLaIiFtEetbkXvFq2cZs+sxYxemTVjBz5XaG98ggo2Tmx1YStzPSHAE7KF4F32exf/5DFOz4kuLjOUx+fTPLNmbX3gdRKjQaZ2qZv5nj8hx2G4ag0lAAACAASURBVLNGdPWbwEzs1xGH3bMZ2Zm7n/3zJ3Lss0Uc2r1dY4yKVmGPMyr4KpRQ4kx5voO5vnHGKzlReKh/J1599VUyMzM1CVI1EkoiNAV4D2gjIguAD4GHavjcr/EcbKb1+/3wjq5klyxXyc4tYOmGbCb268j3MwYxa0RXMtIc7MstYObKip0Ob+A48fWHHFw8hcRGzWh569PY01qWBieloozGmVpW2cGFaQ47yfYExi/Kos+MVRXizLDuGUy/oQv1juxg/7wJuPNzaTHycYoat9cBFxWtIhFn6vygS7Dla8HiTKCpON/BXG+cyUhzgDGYrDdotOZZBndpQatWrcjI0FlmVTNBE6GSKeNteDoTY4FXgZ7GmI9r8lBjzFZjjPbGAwg2uuIvSSrf6RjWPYOLCz7n8IrZJLc5l5a3zCSxUfPS69U9uVmpSNA4Y41gBxfaE4STRcUczXcGjDMAidkb2fGvh0iwJ9Pylpkkl5Tj1wEXFW0iFWdK1OlBl0CxpFWaI+A1mwhjeretMNvjb7n/sO4ZrH7wUjrtfI09K//BNzlF9J3xoQ62qLAImggZYwywzBhz2BizwhjztjHmUC21DQARGSci60VkfU6O/0NGY5Xv8jffEddgp70HS5J83TNqEC16DaH5TVNJSKpf5lpNTm5WKtyiIc7Es0Bxxt+SEy+n2+B0VdzYXD7OpKWlYW9xlmfGuWmbMtd0wEVFk0jGmbo+6OIvlngTmkDXZo3oyuPDupTO9giBl/ufPHmSi68ayMol82jU+0aaDp7AjyeKdeZZhUUoxRIyReQCY8yXVbmxiHwAtPRz6ffGmDdDvY8xZg4wB6Bnz57BChvFlGCbC0M57b28fbkFHD9+nKVLl3LHHXfQs2dP/v7882E/uVmpCLE0zojIOGAcQNu2bavShKgWyhkcDyzKCvl+2bkFuN1uVq1axdVXX80ll1zC+fc+y75jhRVeqwMuKgpVK86o4EKpPhvoWihFVEaNGsWXn35Ik2vuoeH5g0p/H+gcRaWqIpRE6ArgbhH5HjiJpz9ujDHnBXuTMebqmjcvfgWb2ZnYryPjF2VVqKZi8Ewn+6u+0lROcMkll7B161b69OlDhw4dtDS2iiWWxpl4HXCp7AyOYd0zQirR7yXFRYwcOZIlS5bw+eefc+GFF/JQ/0464KJiRbXiDIRn0CVeB1wgeEJT04qRfUfcxef1upFydu8K13TmWdVUKInQgIi3og4KtrlwWPeMgKO0LmOwJwhO9099NXfOLnYtfxxnYT7vvvsuHTp0KL2mJWtVjNA4EwHB4oxXqIc3u/KPkfP64yzJ3sqsWbO44IILAD2LTMWUaseZcAy6xOuASyR8/vnnrF69moceeog3s1P8JkGgM8+q5gImQiKSDNwNnAVsBv5hjCkOx0NF5HrgL0A6sEJEsowx/cJx71gRaPmb90udEeB6msPOyaKf/m8o2PUVOcum06RxGmvXrqVLly6Ra7RSYaZxJrIqizNQ8fBmf7POzqP7OLh4Kq68HBYvXsyNN95Y5roOuKhoFsk4o8LvrbfeYtSoUbRq1Yq777476KyPzjyrmgpWLOFloCeeoDEAmBWuhxpj3jDGtDbGJBljWtS1zgkE31wY7LoIZTYxuwuOk5h2Gu3vmK1JkIpFGmciqLI44zWse0bpa/0tvT219xtM4QmemPNahSRIqRgQsTgDnkEXEdkLXIRn0GVlOO9fl/z973/n+uuv59xzz2XdunU0atQo4KxP4xS7DsCoGgu2NO4cY0wXABH5B/BF7TSpbqhsOUn562kpdgqdLk85W2Nw5uyiXvMzqH/O5aR0vITDJpRVjkpFHY0zEVSVZWv+9hO58o9hS0mlY98h3PXgL7jtinNrpd1KhVlE44wx5g3gjXDes65YtjG7ND4Vf7mIvavmMXjwYBYuXEj9+p6Kt/6W7zrsNqYM0YNUVc0F6z07vX8xxhR7SvCrcKpsOYn3+rKN2UxcvAmn22BcTg6/+2dObvuUVnf8BXvTNogtUdfJqlilcSbCQl22Vn75yfH1b5H76Txa3vwUE0feoCOvKpZpnIlC5ataFjjSSTt/IGMfea40CfImSgVOV+my3Qzdh6jCKFgi1FVEjpf8XQBHyc/eKiuNIt66Osh3dMQ7ejtz5XacboO78AQH33iSU3v+S2rfW0hs0hrw/B+i62RVjNI4YwF/cca7n8i4XRz96J/krX8TR4eLSGzSiolLNgFox0PFKo0zFvAXZ8qX1D55Io+i/d+R3O48Gpx7JZx7JROWfs0zH+7gik7pLN2QXZoouYwpXdqrsUiFS8BEyBjj/6Q9FTGBzvwocLooPnaQg4un4jy6j6aDfusJGCUM2kFRsUnjTO0LFGeG98hg3qf/49Dbs8j/dh0Newyl8ZW/QBJsOF1Gz+tQMUvjTO0LdoYZeJKg3T/s5eDiKRQfO0DGXXOxpaSWXs/OLWBB5p4Kx4jo2UEq3HRjSRQJdOaHTYTcLasoPnGY5iOm4WjXtcxrMnRZnFIqRIHizKuf/0DexnfI//YzGl95J40uuK7Ma/S8DqVUqALFmfGvZWEMFOXs5uDiqbhPnSB92OQySZBXoPriGotUOGkiFEX8lbl1FxVCvWRSLxpBg85XkJjaosx1u010WZxSKmT+4owxBhfQsOdQ6p12NsltKhZF0H2ISqlQBUpWjIHCPZvJef1xJLEeLW+eQb0WZ1bp3hqLVDgFK5+tapmt3AbOvKx32ffiXRQfO4hIQoUkqHGKnZk3dtUpYqVUtRXu3cr+lx+gOO8wkmDzmwTZEoSTp4o5fdIK+sxYxbKN2aXXlm3Mps+MVX6vKaXqnmUbs0kIUpCiYOd6Euo3puWtsypNgsrfxV/5f6VqQmeELOJvE6H3/A5j3OR+8m+OZy7BcUZPEhwNy7w3I83B2klX+rutUioOVbbpuLpObl/LoeVPk9ioGaa4yO9r6tezUVTsJrfAU3ir/Fr/QPsAdIBGqdgSjjjj3RtU/jwyYwzuguPYUlJJu+x2Ui8aQUJS/aD3cthtDO+RwUfbcsIe+5Ty0kSohqoTOAJtIhQBt9PJoXdmk7/1Exp060+Ta+5BEsru8/Sdco5UB0kpFR2CbTquznfdJkKx203e+jc5uuofJLXqSPrwP1RYo+8dcOkzY1WF5XTeDcvev/u7pnFIqdgRrjjjb2+Qcbs4uuof5G9fw2m3P4utQWMkQBJkE8FtjPZnVK3RRKgGqhs4Am0iBMhdt5D8rZ+QdtlYGvUajr/zDrzrY8PdQVJKRZ9A8aK6yUbvMxrz3uJ5HF01l5QOF9N08AQS/r+9e4+Oqrz3P/7+EkKSBhCqKHJTf4KhFFAkalyxekDkKghBikWUH4jWas8RLyjKOmjLUREvR6y1Wis/0IJoQcGfgCIq96Y1VZAgSIUePAQUxIaLILk9549MOBNIJpMwM3tmz+e1VtbKzOzZz5dAPuzv3s9+JjWt2jbB009qm+sf6oZl3cwsklhqy5kJr63n8Xc/D/sk7/EnTSpKj7KvaiXK7GtolHniogjVtneOf0wb1LA/hEgDqBE6CQ09QAl1kHBKzrWktcniBx0vrvH14AOUusbX1SKRxNeQRqTK8RnQq3MrPv5yP5ldrqCi9AjNL87DrBGdTs/kcElFjVlR9flCx6s6IRPqNRFJDKHyJJyTrFUnZoOVHznI3gVTOVq0mZa9x9P8oqF11qHskFjTYgknoaEHKMf/oh/dvZU9839FRckRGjXJqNYEpaYYLTJSMSqnqjya1+1YEIUavyqUioqP4PjfINONzCKJpbYDg7oOGI7PgC+LdvPM1Ac4fOQIjdKbcsol12JW+V/A4ZIK1k7qzT+mDWLtpN7VDnYm9ssiI7X69NyqEzKhXhORxFFXngRPh61JTSdmi1e9zNGvtnLaNfcda4JG53So9SM/9OHw4gU1QiehoQcowQcPh//+F75+9X5KvvmS8sP7q21nwMiL2rP+wb41HqCEGj/U1SIRSRwNbTaCM6B0306++uM9HPr0PUq+3n7CtqFO3gzt0ZZH87rRtkXGCSdkQr0mIomjppw5XrjTYV1goYSW/zKW1j+bRmbny4DKhVf+Y2i32pfWRtP6JfY0Ne4kTOyXVe0eHQjvAKXqF338fQ+zb/kLNGl9LqcPn0JKZstq2zngwy176z1+r86t+GP+lzW+R3P3RRJLVV7Ud5pr1e/69zs3sXfBf0CjRpzxs0dIa3NiPtV18qaq6Ql2/LS7/xx5gQ5iRBJUcM7UNN0VQudE1RTaI9v/xoG/vkGrvH+nUdoPSGvb+dg2h0vKq217PH04vHhBjdBJOP4ApcUPUnEO7gzj5sJtH8xj33u/o2mnS2h59UQaNUmvcbu6ztQGj191D8CCv9U+/U3zb0UST3AjUtWA3Pna+pBNUZsWGfz9oxXsXTSNxs1bcfqIX5Ha8swTtmvIhzJroRYR/6nKmeN/v6H6Sd6a7j+e2C+LX0x5kj2LZ5Da6ixc6RE47rim6vijoSeRRaJBjdBJqi046jowyMvLY9++fdDzp7z6URHuhC0q1fdMbe60D06YEldFQSOS2OqTMxP7ZXHXjm0cOqcHpw6cQEpG8xr3mdmkcb2bl0ivZCci8SPUVeiaMuiOeZ+wf9089q+ZQ/pZF9Bq2AOkpP2g2nFN8PFHQ69yi0SDGqEICefA4JtvvuGFF17gR/1v5Mn3/sGuilwaFeyqtQlqSOMS6gqS5u6LJLZwcqa8vJzXX3+d9KyfcMqZZ1ExfErIfe4PfFBqfZzMSnYiEv9qmg4LNWfQ/rVz2b/2VTK79ubU/v9Kk9QmjLy4fcgPQq1t/yKxpkYoQuo6MNi2bRsDBgzgv3Z8SdsdP8T9sAPACZ++HKwhjUuoubcKHZHEVlfOzFv3d24bP4Z/bv4zZ1z3MOlnnV/nPhsyXbauJbVFxJ9qyqDMH/fGUlJpnjMCM6O0wvH2ht2sf7CvBxWK1I9WjYuQUCu45efnk5OTw7fffkvW2OnHmqBQGtq4aDlbEf8KlTOzlq9n7LWD+OeWv/DDq34RVhPU0GxQzogkp6oMKv/unxSvm4dzjtSWZ3LKpT+t9gHwxUdK9XEdkhA8aYTM7HEz22Jmn5rZm2bWwos6Iqm2A4PL0/6LXr160Tg9kw7/9ykOtji3zn2dzAGFlrMV8a/acuZn56Vw608HcPSbL2mVN5lmF9b+yey1fS5ZfShnRJLTxH5ZpBzYzVev3MOB/D9Rtm9nrdvq4zokEXg1Ne494H7nXJmZPQbcD9znUS0RUdvNfy32p/NWl/MpuWIC3zZuVud+2kbgpkHNvRWpPOECDAZKgG3AWOdcsbdVnZzacib9642UlZZyxs+mkXZmp5D7yExrHJEpK8oZkeRzxvdfsvfVe6HMccZ1j5B6Wvtat9U9g5IIPGmEnHPLgh7mA9d6UUekVR0YlJeXs2LFCq7s0RZoyynXPsyu/d/X+X4D1k7qHfU6RZKE7064QPUGZNu2bZx7bmXOZN/7Cl99V1Hn+3VwIhJZfjzpUpO33nqLkSNH0q5dO5YuXUrHjh1Z+EkRd72+nooabnfWPYOSCOLhHqFxwFKvi4iUw4cPc+2119KnTx8+/vhjAHaH0QSBQkMkkpxzy5xzZYGH+UA7L+uJJOccjz32GFlZWaxYsQKASVd3r/OT4UE5IxIF7wFdnXPdga1UnnTxnczMTLKzs1m3bh0dO3YEKk/MPPXTC3TPoCSsqDVCZrbczApr+LomaJvJQBkwJ8R+bjGzAjMr2Lt3b7TKjYg9e/bQu3dvFi1axIwZM7jwwguB8A48FBoiURXyhEsi5UxZWRm33XYbkyZNYsSIEeTk5AAn3rfTIiOV1BSr9l7ljEjk+fmkS0VFBatXrwbgyiuvZNWqVbRq1araNrpnUBKZuRDLN0d1YLMxwK3Alc65w+G8Jzs72xUUFES3sAbaunUrAwYMYNeuXcydO5dhw4Yde62mT2lObWQ0TW9M8eFSfZiYxA0z+5tzLtvrOsJlZsuB1jW8NNk5tyiwzWQgG8hzYQRePOfMoUOHGDlyJEuWLGHSpEk8/PDDNGpU+/msmj4BXjkjXku0nKkPM/v/wGvOuT+G2i6ec6bK0aNHGTduHHPnzuWjjz4iO9uXf2XiU+HmjCf3CJlZfyrn6l8RbhMU71asWMHBgwdZsWIFl1xySbXX9CnKItHhnOsT6vXACZerqTzh4s1ZnwiaP38+77zzDs8//zw///nP69xeCxqIREY9TrrUOsvFzG4BbgHo0KHuj9HwUnFxMXl5eXz44Yc88sgj9OzZ0+uSRKLCkytCZvYFkAbsCzyV75y7ta73xeMZlD179nD66acDsG/fPk499VSPKxJpOD+dqQ2ccHmKyhMuYc93i8ecKS0tJTU1FecchYWFdOvWzeuSRBrMTzlTpb6zXOIxZ6rs3LmTAQMGsGXLFmbOnMkNN9zgdUki9RZuzniyWIJzrqNzrr1z7oLAV51NULxxzvHEE0/QsWNHCgsLAdQEicSXZ4FmwHtmtt7Mnve6oIZYsWIF5513Hps2bcLM1ASJxJmgWS5D/DDLZenSpezYsYOlS5eqCRLf8+pzhBJaWVkZd9xxB8899xwjRow4tnqKiMQP51zC/2LOmTOHsWPH0rFjR5o2bep1OSJSs2epnOXynplBmLNc4s3Bgwdp1qwZN998M4MGDaJNmzZelyQSdfGwfHZC+e677xg2bBjPPfccEydOZN68eaSnp3tdloj4iHOORx55hNGjR5Obm8vatWs566yzvC5LRGrgh1kuc+fO5ZxzzmHDhg0AaoIkaagRqqdnnnmGJUuW8Nvf/pbp06eHXLFJRKQhZs2axeTJkxk1ahTvvPMOLVu29LokEfEh5xzTp0/n+uuvp2vXrnG/iINIpGlqXJicc5gZ99xzD5dffjm5ublelyQiPjVq1CjKy8u56aabCEy1ERGJqPLyciZMmMCzzz7LyJEjmT17NmlpaV6XJRJTupwRhpUrV5Kdnc3XX39NamqqmiARibhdu3Zx3XXX8e2335KWlsb48ePVBIlI1Lzwwgs8++yz3H333cydO1dNkCQlNUJ1mDt3Ln379uXw4cN8//33XpcjIj60adMmcnJyePvtt/nss8+8LkdEksDNN9/MggULeOKJJzTNX5KW/uXXwjnHo48+yvXXX09OTg7r1q3TzcoiEnEffPABubm5lJWVsXr1ai677DKvSxIRn9q+fTsDBw5kz549pKamkpeX53VJIp5SI1SLp59+mgceeIBRo0axbNky3awsIhH39ttv079/f9q1a0d+fj49evTwuiQR8amCggIuvfRS8vPz2bFjh9fliMQFLZZQixtvvBHnHBMmTNAlYxGJiuzsbEaNGsXTTz9NixYtvC5HRHxqyZIljBgxglatWrFy5Uo6d+7sdUkicUFH+EF27drF7bffztGjRzn11FO566671ASJSESVlpbyzDPPUFZWRuvWrZk1a5aaIBGJmkWLFjFkyBCysrLIz89XEyQSREf5AYWFheTk5DB79mwKCwu9LkdEfOjAgQNcffXV3HHHHSxZssTrckQkCeTm5jJ+/HhWrlxJ69at69x+4SdF5E77gHMmLSZ32gcs/KQoBlWKeEONECferNyzZ0+vSxIRnykqKuInP/kJ77//Pi+99BJDhgzxuiQR8anS0lKeeuopSkpKOO2003j++edp1qxZne9b+EkR97+xkaLiIzigqPgI97+xUc2Q+FbSN0J/+tOf6N+/P+3bt9fNyiISFRs3biQnJ4ft27ezePFixo0b53VJIuJTBw8eZPDgwdx9990sXry4Xu99/N3POVJaXu25I6XlPP7u55EsUSRuJH0j1KVLFwYPHsyaNWvo0KGD1+WIiA+VlJSQmZnJmjVr6Nevn9fliIhP7d69myuuuILly5fz4osvMmzYsHq9f1fxkXo9L5LokrIRKi0tZc6cOTjn+PGPf8yCBQt0s7KIRFxBQQEAPXv2ZNOmTZx//vkeVyQifrVlyxYuvfRSPv/8c9566y3Gjx9f7320aZFRr+dFEl3SNUIHDhxg0KBBjB49mnXr1nldjoj4kHOOhx56iIsuuoiFCxcCkJKS4nFVIuJnpaWlpKWlsXLlSgYOHNigfUzsl0VGavWsykhNYWK/rEiUKBJ3kupzhHbu3MmgQYP47LPPmDlzJrm5uV6XJCI+U1JSwi233MLs2bMZM2ZMgw9IRETCsWHDBrp37063bt3YtGkTjRs3/NBuaI+2QOW9QruKj9CmRQYT+2Ude17Eb5KmEdqwYQODBg3iwIEDLFmyhKuuusrrkkTEZ/bv38/w4cN5//33eeihh5gyZQpm5nVZIuJTM2bM4M477+Tll19m9OjRJ9UEVRnao60aH0kaSdMI7dixg5SUFNasWUP37t29LkdEfGjVqlWsWrWKWbNmMWbMGK/LERGfqqio4N577+XJJ59k6NChDB8+3OuSRBKS7xuhrVu3ct555zFkyBD69u1Lenq61yWJiM8cOnSIpk2bMnjwYL744gutQCkiUXP06FHGjBnDa6+9xu23386MGTN0D6JIA3myWIKZTTWzT81svZktM7M2kR7DOceUKVPo0qULf/7znwHUBIkkmVhkzbJlyzj77LNZtWoVgJogEYmqtWvXMn/+fKZPn85vfvMbNUEiJ8GrVeMed851d85dALwNTInkzktKShgzZgxTp05lzJgxZGdnR3L3IpI4opo1M2fOZODAgbRr145zzz03krsWEanm6NGjAPTu3ZvNmzczceJE3YMocpI8aYSccweCHmYCLlL7Li4uZsCAAbzyyiv8+te/5g9/+AOpqamR2r2IJJBoZU3VFeebbrqJPn36sGrVKtq21c3FIskoFleeN2zYQFZWFu+++y4AnTp1ivQQIknJs3uEzOxh4EZgP9ArUvt99dVXWb16NS+//DI33HBDpHYb0sJPirTUpEicikbWzJ8/n6lTp3LTTTfxu9/9LmYnW5Q1InHpcefcvwOY2b9ReeX51kjtfPny5eTl5dG8eXPatIl4jyWS1My5iF2Mqb5js+VA6xpemuycWxS03f1AunPuwVr2cwtwC0CHDh167tixI+S4zjk2b95Mly5dGlx7fSz8pIj739jIkdLyY89lpKbwaF43HaBIwjGzvznnEmouaSSypr45U1FRwcKFCxk2bFjMpqYoa8QvEjFnwhXImQ7OuV+E2i47O9sVFBTUub9XXnmFcePG0blzZ5YuXUq7du0iVaqIr4WbM1FrhMJlZmcBi51zXevaNtzgiKXcaR9QVHzkhOfbtshg7aTeHlQk0nA+P0AJK2viMWdAWSP+4cecOf7Ks3Nub6jtw8mZ1atXc/nll9OrVy/efPNNTjnllMgVLOJz4eaMV6vGBU9uHQJs8aKOSNhVw4FJqOdFJHaUNSISCWa23MwKa/i6BsA5N9k51x6YA/yyln3cYmYFZlawd2/IPgmAyy67jBdffJGlS5eqCRKJEq/uEZpmZllABbCDCM6ljbU2LTJqPEvbpkWGB9WIyHGUNSJy0pxzfcLcdC6wGDhhCq5z7vfA76HyilBdOzIzxo8fX58yRaSevFo1brhzrmtgWdvBzrkiL+qIhIn9sshIrb6Gf0ZqChP7ZXlUkYhUUdaISLT56cqzSLLxbNU4v6i6SVkrOYlINClrROKWb648iyQbNUIRMLRHWx2MiEjUKWtE4o9zbrjXNYhIw3gyNU5ERERERMRLaoRERERERCTpqBESEREREZGko0ZIRERERESSjjlX51L2ccPM9lK5IktdTgO+iXI5qiH+x1cN9a/hLOdcq2gXE8+UM6ohAcdPtBqUM8oZ1ZCYNXg9fn1qCCtnEqoRCpeZFTjnslWDtzV4Pb5qiK8a/CYefqaqIT5q8Hp81eBf8fAzVQ2qIV7Gj0YNmhonIiIiIiJJR42QiIiIiIgkHb82Qr/3ugBUQzyMD6qhSjzU4Dfx8DNVDZW8rsHr8UE1+FU8/ExVQyXV4P34EOEafHmPkIiIiIiISCh+vSIkIiIiIiJSK982QmY21cw+NbP1ZrbMzNrEePzHzWxLoIY3zaxFLMcP1DDCzDaZWYWZxXSVDzPrb2afm9kXZjYplmMHxp9pZnvMrDDWYwfV0N7MPjSzzYG/hzs8qCHdzP5qZhsCNfwq1jX4mdc5E6jB06xRzihnlDPRpZw5VoMnWaOc8XnOOOd8+QU0D/r+34DnYzx+X6Bx4PvHgMc8+Bn8CMgCVgDZMRw3BdgG/B+gCbAB6BLjP/vlwIVAYax/7kE1nAlcGPi+GbDVg5+DAU0D36cCfwFyvPqZ+O3L65wJjOtp1ihnlDPKmaj/fJM+ZwLjxjxrlDPHavBtzvj2ipBz7kDQw0wgpjdDOeeWOefKAg/zgXaxHD9Qw2bn3OexHhe4GPjCObfdOVcCzAOuiWUBzrlVwLexHLOGGnY75z4OfH8Q2Ay0jXENzjl3KPAwNfClGwMjxOucCdTgadYoZ5QzypnoUs4cq8GLrFHO4O+c8W0jBGBmD5vZfwPXA1M8LGUcsNTD8WOtLfDfQY93EuNfmHhjZmcDPag8gxHrsVPMbD2wB3jPORfzGvwsjnIGkitrlDPHUc74l3LGM8qZ4/gtZxK6ETKz5WZWWMPXNQDOucnOufbAHOCXsR4/sM1koCxQQ8SFU4MHrIbnkvbsoJk1BRYAE447sxcTzrly59wFVJ7Bu9jMusa6hkTmdc6EU0Ngm6hljXIm/ilnEptyJvwaYkw5E8SPOdP45MvyjnOuT5ibzgUWAw/GcnwzGwNcDVzpApMaI60eP4NY2gm0D3rcDtjlUS2eMrNUKkNjjnPuDS9rcc4Vm9kKoD/g2U2XicbrnAmnhmhnjXImvilnEp9yJrwaPKCcCfBrziT0FaFQzKxT0MMhwJYYj98fuA8Y4pw7HMux48BHQCczO8fMeMcE7AAAA2hJREFUmgDXAW95XFPMmZkBLwGbnXNPeVRDq6rVfcwsA+hDjH8X/MzrnAnUkKxZo5xBOZMMlDOeUs7g75zx7QeqmtkCKlcXqQB2ALc654piOP4XQBqwL/BUvnPu1liNH6hhGPAboBVQDKx3zvWL0dgDgaepXHFlpnPu4ViMGzT+q8C/AKcBXwMPOudeinENlwGrgY1U/jsEeMA5tySGNXQHZlP599AIeN059+tYje93XudMoAZPs0Y5o5xRzkSXcuZYDZ5kjXLG3znj20ZIRERERESkNr6dGiciIiIiIlIbNUIiIiIiIpJ01AiJiIiIiEjSUSMkIiIiIiJJR42QiIiIiIgkHTVCSc7MTjWz9YGvr8ysKOhxkwiNMTBon4fM7PPA9/+vnvtpbGb3RqImEYkd5YyIRJtyRhpCy2fLMWb2EHDIOffEcc8blf9WKmp8Y/3GWAP80jm3vgHvTQd2OudOO9k6RMQbyhkRiTbljIRLV4SkRmbW0cwKzex54GOgvZkVB71+nZn9IfD9GWb2hpkVmNlfzSynHuOkmtnTgfdtMLOxgefbm9nawJmWjWZ2CTANaNGQsy8iEn+UMyISbcoZCaWx1wVIXOsCjHXO3Wpmof6tPANMd87lm9nZwNtA1zDH+AWVZ0UuDpwh+YuZLQNuBN5wzj1pZilABjAJGO2cu6CBfx4RiT/KGRGJNuWM1EiNkISyzTn3URjb9QGyKq84A9DSzDKcc0fCeG9foKOZjQ48bg50BP4KPGdmmcBC59yngWAREX9RzohItClnpEZqhCSU74K+rwAs6HHwL7EBFzvnShowhgE/d86tPOEFs17AIGCemf0KWNSA/YtIfFPOiEi0KWekRrpHSMISuLHwn2bWycwaAcOCXl4O3F71wMzqc6n3XeC2qkvVZvYjM0sPXJLe7Zx7AXgZ6AEcBRoHLi2LiM8oZ0Qk2pQzEkyNkNTHfcA7wPvAzqDnbwdyzexTM/sMuLke+3wO+BL4xMwKgWeBFOAqYIOZfQIMAH7rKpc4nAVs1M2FIr6lnBGRaFPOCKDls0VEREREJAnpipCIiIiIiCQdNUIiIiIiIpJ01AiJiIiIiEjSUSMkIiIiIiJJR42QiIiIiIgkHTVCIiIiIiKSdNQIiYiIiIhI0lEjJCIiIiIiSed/AH/0rYVESRTaAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1008x864 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Set up the axes with gridspec\n",
    "fig = plt.figure(figsize=(14, 12))\n",
    "grid = plt.GridSpec(3, 3, hspace=0.2, wspace=0.3)\n",
    "plt_linear = fig.add_subplot(grid[0, 0])\n",
    "plt_ridge  = fig.add_subplot(grid[0, 1])\n",
    "plt_lasso  = fig.add_subplot(grid[0, 2])\n",
    "\n",
    "# Linear Regression 在测试集上的性能\n",
    "plt_linear.scatter(y_test, y_test_pred_lr)\n",
    "plt_linear.plot([-3, 3], [-3, 3], '--k')   \n",
    "plt_linear.axis('tight');\n",
    "plt_linear.set_xlabel('True Test');\n",
    "plt_linear.set_ylabel('Predicted Test');\n",
    "plt_linear.set_title('Linear');\n",
    "\n",
    "# Ridge Regression 在测试集上的性能\n",
    "plt_ridge.scatter(y_test, y_test_pred_ridge)\n",
    "plt_ridge.plot([-3, 3], [-3, 3], '--k')   \n",
    "plt_ridge.axis('tight');\n",
    "plt_ridge.set_xlabel('True Test');\n",
    "plt_ridge.set_ylabel('Predicted Test');\n",
    "plt_ridge.set_title('Ridge');\n",
    "\n",
    "# Lasso Regression 在测试集上的性能\n",
    "plt_lasso.scatter(y_test, y_test_pred_lasso)\n",
    "plt_lasso.plot([-3, 3], [-3, 3], '--k')   \n",
    "plt_lasso.axis('tight');\n",
    "plt_lasso.set_xlabel('True Test');\n",
    "plt_lasso.set_ylabel('Predicted Test');\n",
    "plt_lasso.set_title('Lasso');\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上述图中观测到：\n",
    "\n",
    "1. Linear 和 Ridge 模型的散点图相似，说明这两种模型在测试集上的性能相似。\n",
    "\n",
    "\n",
    "2. Lasso 模型对大多数情况的估计和前两种模型类似，但对于较小的真值，Lasso 模型可能较其他两种模型性能稍好。\n",
    "\n",
    "\n",
    "3. 三个模型在测试集的 RMSE 都为 0.4 左右，也侧面反映了它们在测试集上的性能相似。"
   ]
  }
 ],
 "metadata": {
  "_change_revision": 0,
  "_is_fork": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
